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Basic Concepts 


11 OVERVIEW: SYSTEM LIFE CYCLE 


We assume that our readers have a strong background in programming, typically 
attained through the completion of an elementary programming course. Such an 
initial course usually emphasizes mastering the syntax of a programming 
language (its grammar rules) and applying this language to the solution of 
several relatively small problems. In this text we want to move beyond these 
rudiments by providing the tools and techniques necessary to design and imple- 
ment large-scale computer systems. We believe that a solid foundation in data 
abstraction and encapsulation, algorithm specification, and performance analysis 
and measurement provides the necessary methodology. In this chapter, we will 
discuss each of these areas in detail. We will also briefly discuss recursive pro- 
gramming because many of you probably have only a fleeting acquaintance with 
this important technique. However, before we begin, we want to place these 
tools in a context that views programming as more than writing code. Good pro- 
grammers regard large-scale computer programs as systems that contain many 
complex interacting parts. As systems, these programs undergo a development 
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process called the system life cycle. This cycle consists of requirements, 
analysis, design, coding, and verification phases. Although we will consider 
them separately, these phases are highly interrelated and follow only a very 
crude sequential time frame. The References and Selected Readings section lists 


several sources on the system life cycle and its various phases that will provide 
you with additional information. 


(1) Requirements. All large programming projects begin with a set of 
specifications that define the purpose of the project. These requirements 
describe the information that we, the programmers, are given (input) and the 
results that we must produce (output). Frequently the initial specifications are 


defined vaguely, and we must develop rigorous input and output descriptions that 
include all cases. 


(2) Analysis. After we have delineated carefully the system's requirements, the 
analysis phase begins in eamest. In this phase, we begin to break the problem 
down into manageable pieces. There are two approaches to analysis: bottom-up 
and top-down. The bottom-up approach is an older, unstructured strategy that 
places an early emphasis on the coding fine points. Since the programmer does 
not have a master plan for the project, the resulting program frequently has many 
loosely connected, error-ridden segments. Bottom-up analysis is akin to con- 
structing a building by first designing specific aspects of the building such as 
walls, a roof, plumbing, and heating, and then trying to put these together to con- 
struct the building. The specific purpose to which the building will be put is not 
considered in this approach. Although few of us would want to live in a home 
constructed using this technique, many programmers, particularly beginning 
ones, believe that they can create good, error-free programs without prior plan- 
ning. 

Tn contrast, the top-down approach begins by developing a high-ievel plan 
for dividing the program into manageable segments. This plan is subsequently 
refined to take into account low-level details. This technique generates diagrams 
that are used to design the system. Frequently, several alternate solutions to the 
programming problem are developed and compared during this phase. The top- 


down approach has been the preferred approach for developing complex 
software systems. 


(3) Design. This phase continues the work done in the analysis phase. The 
designer approaches the system from the perspectives of the data objects that the 
program needs and the operations performed on them. The first perspective 
leads to the creation of abstract data types, whereas the second Tequires the 
Specification of algorithms and a consideration of algorithm design strategies. 
For example, suppose that we are designing a scheduling system for a university. 
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Typical data objects might include students, courses, and professors. Typical 
operations might include inserting, removing, and searching within each object 
or between them, That is, we might want to add a course to the list of university 
courses or search for the courses taught by a specific professor. 

Since the abstract data types and the algorithm specifications are 
language-independent, we postpone implementation decisions. Although we 
must specify the information required for each data object, we ignore coding 
details. For example, we might decide that the student data object should 
include name, social security number, major, and phone number. However, we 
would not yet pick a specific implementation for the list of students. As we will 
see in later chapters, there are several possibilities, including arrays, linked lists, 
or trees. By deferring implementation issues as long as possible, we not only 
create a system that could be written in several programming languages, but we 
also have time to pick the most efficient implementations within our chosen 
language. 


(4) Refinement and coding. In this phase, we choose representations for our 
data objects and write algorithms for each operation on them. The order in 
which we do this is crucial because a data object’s representation can determine 
the efficiency of the algorithms related to it. Typically this means that we should 
write the algorithms that are independent of the data objects first. 

Frequently at this point we realize that we could have created a much 
better system. Perhaps we have spoken with a friend who has worked on a simi- 
lar project, or we realize that one of our alternate designs is superior. If our ori- 
ginal design is good, it can absorb changes easily. In fact, this is a reason for 
avoiding an early commitment to coding details. If we must scrap our work 
entirely, we can take comfort in the fact that we will be able to write the new 
system more quickly and with fewer errors. 


(5) Verification, This phase consists of developing correctness proofs for the 
program, testing the program with a variety of input data, and removing errors. 
Each of these areas has been researched extensively, and a complete discussion 
is beyond the scope of this text. However, we summarize briefly the important 
aspects of each area. 


Correctness proofs: Programs can be proven correct using the same techniques 
that abound in mathematics. Unfortunately, these proofs are very time- 
consuming and difficult to develop for large projects. Frequently, scheduling 
constraints prevent the development of a complete set of proofs for a large sys- 
tem. However, selecting algorithms that have been proven correct can reduce 
the number of errors. In this text, we will provide you with an arsenal of algo- 
rithms, some of which have been proven correct using formal techniques, that 
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you may apply to many programming problems. 


Testing: Since our algorithms need not be written in a specific programming 
language, we can construct our correctness proofs before and during the coding 
phase. Testing, however, requires the working code and sets of test data. This 
data should be developed carefully so that it includes all possible scenarios. Fre- 
quently. beginning programmers assume that if their program ran without pro- 
ducing a syntax error, it must be correct. Little thought is given to the input data, 
and usually only one set of data is used. Good test data should verify that every 
piece of code runs correctly. For example, if our program contains a switch 
statement, our test data should be chosen so that we can check each case within 
the switch statement. - 

Initial system tests focus on verifying that a program runs correctly. 
Although this is a crucial concern, a program's running time is also important. 
An error-free program that runs slowly is of little value. Theoretical estimates of 
Tunning time exist for many algorithms. We will derive these estimates as we 
introduce new algorithms. In addition, we may want to gather performance esti- 


mates for portions of our code. Constructing these timing tests is also a topic 
that we pursue later in this chapter. 


Error removal: If done properly, the correctness proofs and system tests will 
indicate erroneous code. The ease with which we can remove these errors 
depends on the design and coding decisions made earlier. A large undocu- 
mented program written in “spaghetti” code is a programmer's nightmare. When 
debugging such programs, each corrected error possibly generates several new 
errors. On the other hand, debugging a well-documented program that is divided 
into autonomous units that interact through parameters is far easier, especially if 
each unit is tested separately and then integrated into the system. 


1.2 OBJECT-ORIENTED DESIGN 


Object-oriented design represents a fundamental change from the structured pro- 
gramming design method. The two approaches are similar in that both believe 
that the way to develop a complex system is by using the philosophy of divide- 
and-conquer: that is, break up a complex software design project into a number 
of simpler subprojects, and then tackle these subprojects individually. The two 
approaches disagree on how a project should be decomposed. 
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1.2.1 Algorithmic Decomposition Versus Object-Oriented Decomposition 


Traditional programming techniques have used algorithmic decomposition. 
Algorithmic or functional decomposition views software as a process. It decom- 
poses the software into modules that represent steps of the process. These 
modules are implemented by language constructs such as procedures in Pascal, 
subprograms in Fortran, and functions in C. The data structures required to 
implement the program are a secondary concem, which is addressed after the 
project has been decomposed into functional modules. 

Object-oriented decomposition views software as a set of well-defined 
objects that model entities in the application domain. These objects interact with 
each other to form a software system. Functional decomposition is addressed 
after the system has been decomposed into objects. 

The principal advantage of object-oriented decomposition is that it 
encourages the reuse of software. This results in flexible software systems that 
can evolve as system requirements change. It allows a programmer to use 
object-oriented programming languages effectively. Object-oriented decomposi- 
tion is also more intuitive than algorithm-oriented decomposition because 
objects naturally model entities in the application domain. 


1.2.2 Fundamental Definitions and Concepts of Object-Oriented Program- 
ming 


Before proceeding, we provide definitions of basic terminology used in the 
object-oriented paradigm. Our definitions are adapted from Object- Oriented 
Design with Applications, by Grady Booch. 


Definition: An object is an entity that performs computations and has a local 
state. It may therefore be viewed as a combination of data and procedural ele- 
ments. 


Definition: Object-oriented programming is a method of implementation in 
which 

(1) Objects are the fundamental building blocks. 

(2) Each object is an instance of some type (or class). 


(3) Classes are related to each other by inheritance relationships. (Program- 
ming methods that do not use inheritance are not considered to be object- 
oriented.) 
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Definition: A language is said to be an object-oriented language if 
(1) It supports objects. 


(2) It requires objects to belong to a class. 
(3) It supports inheritance. 


A language that supports the first two features but does not support inheritance, 


is an object-based language. C++ is an object-oriented language while 
JavaScript is an object-based language. 


1.2.3. Evolution of Programming Languages and History of C++ 


Higher order programming languages may be classified into four generations. 


(1) First Generation Languages: An example of this is FORTRAN. The salient 
feature of these languages is their ability to evaluate mathematical expres- 
sions. 

(2) Second Generation Languages: Examples of these include Pascal and C, 
The emphasis of these languages is on effectively expressing algorithms. 

(3) Third Generation Languages: An example of this is Modula, which intro- 
duced the concept of abstract data types. 

(4) 


Fourth Generation Languages (Object-Oriented languages): Examples 
include Smalltalk, Objective C, and C++. These languages emphasize the 


expression of the relationship between abstract data types through the use 
of inheritance. 


C++ was designed by Bjarne Stroustrup of AT&T Bell Laboratories in the early 
1980s. It was designed to incorporate the object-oriented paradigm into the C 
programming language. C is widely-used in industry because it is: 


(1) Efficient: it has a number of low-level features, which utilize hardware 
better than other languages. 

(2) Flexible: it can be used to solve problems in most application areas. 
Q) 


Available: compilers for C are readily available for most computers. 
Other influences on the design of C++ were Simula67 and Algol68. 
There are a number of features of C++ that improve on C, but do not 


implement data abstraction or inheritance. These are discussed later in this 
chapter. ‘The C++ class, which implements data abstraction, is discussed in 
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Chapter 2. C++ functions and classes can be made more powerful by augmenting 
them with templates. Templates are discussed in Chapter 3. Simple inheritance 
in C++ is also discussed in Chapter 3. Advanced concepts on inheritance and 
polymorphism are discussed in Chapter 4. 


1.3 DATA ABSTRACTION AND ENCAPSULATION 


The notions of abstraction and encapsulation are commonly used in human- 
machine interaction and are widely prevalent in the technology-oriented world 
we live in. Consider, for example, the digital video player (DVD player), which 
is an integral part of many households today. We make two observations about 
how DVD players are packaged and used. 


(1) All our interactions with a DVD player are made through buttons on the con- 
tro] panel (or remote control) such as PLAY, STOP and PAUSE. The DVD 
player is packaged so that we cannot directly interact with the circuitry inside. 
So, the internal representation of the DVD player is hidden from the user. This 
is the principle of encapsulation. 


(2) The instruction manual that accompanies the DVD player tells us what the 
DVD player is supposed to do if we press a particular button. It does not tell us 
how this function is implemented inside the DVD player; it does not, for exam- 
ple, explain the sequence of electronic and mechanical events that result when 
the PLAY button is pressed. So, the instruction manual makes a clear distinction 
between what the DVD player does and how it does it. This distinction is the 
principle of abstraction. 


These same principles may be applied to the packaging or organizing of data in a 
computer program, resulting in the concepts of data encapsulation and data 
abstraction, which are defined below. 


Definition: Data Encapsulation or Information Hiding is the concealing of the 
implementation details of a data object from the outside world. 


Definition: Data Abstraction is the separation between the specification of a data 
object and its implementation. 


We will see later that these concepts are of great importance in software 
development because they result in better quality programs and more efficient 
programming techniques. These. in turn, reduce the number of person-hours 
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Tequired to develop a software system, and hence reduce the total cost of the 
final software product. 

First, however, we shall explore the fundamental data types of C++. These 
include char, int, float, and double. Some of these data types may be modified 
by keywords short, long, signed, and unsigned. The modifiers short and long 
specify the amount of storage allocated to the fundamental type they modify, 
The modifiers signed and unsigned specify whether the most significant bit in 
the binary representation of an integer represents a sign bit or not. C++ also sup- 
ports types derived from the fundamental data types listed above. These include 
pointer and reference types. Ultimately, the real world abstractions we wish to 
deal with must be represented in terms of these data types. In addition to these 
types, C++ helps us by providing three mechanisms for grouping data together. 
These are the array, the struct, and the class. Arrays are collections of elements 
of the same basic data type; structs and classes are collections of elements 
whose data types need not be the same. 


All programming languages provide at least a minimal set of predefined 
data types, plus the ability to construct new, or user-defined types. 


Definition: A data type is a collection of objects and a set of operations that act 
on those objects. 0 


Whether your program is dealing with predefined data types or user-defined data 
types, these two aspects must be considered: objects and operations. For exam- 
ple, the data type int consists of the objects {0, +1, -1, +2, -2, --- , MAXINT, 
MININT}, where MAXINT and MININT are the largest and smallest integers 
that can be represented by an int on your machine. The operations on integers 
are many and would certainly include the arithmetic operators +, —, *, and /. 


There is also testing for equality/inequality and the operation that assigns an 
integer to a variable. 


Definition: An abstract data type (ADT) is a data type that is organized in such 
a way that the specification of the objects and the specification of the operations 


on the objects is separated from the representation of the objects and the imple- 
mentation of the operations. O 


Throughout this text, we will emphasize the distinction between 
specification and implementation. To help us do this, we will typically begin 
with an ADT definition of the object that we intend to study. This will permit the 
teader to grasp the essential elements of the object, without having the discus- 
sion complicated by the representation of the objects or by the actual implemen- 
tation of the operations. Once the ADT definition is fully explained, we will 
Move on to discussions of representation and implementation. These are quite 
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important in the study of data structures. To help us accomplish this goal, we 
introduce a notation for expressing an ADT. This notation is independent of the 
syntax of the C++ language. Later we shall see how the syntax of the C++ class 
can be used to express an ADT. 


Example 1.1 [Abstract data type NaturalNumber): As this is the first example of 
an ADT, we will spend some time explaining the notation. ADT 1.1 contains the 
ADT definition of NaturalNumber. The definition begins with the name of the 
abstract data type. There are two main sections in the definition: the objects and 
the functions. The objects are defined in terms of the integers, but we make no 
explicit reference to their representation. The function definitions are a bit more 
complicated. First, the definitions use the symbols x and y to denote two ele- 
ments of the set NaturalNumber, whereas TRUE and FALSE are elements of the 
set of Boolean values. In addition, the definition makes use of functions that are 
defined on the set of integers, namely, plus, minus, equal, and less than. This is 
an indication that in order to define one data type, we may need to use operations 
from another data type. For each function, we place the result type to the left of 
the function name anda definition of the function to the right. The symbols "::=" 
should be read as "is defined as.” 

The first function, Zero, has no arguments and returns the natural number 
zero, The function Successor{x) returns the next natural number in sequence. 
Notice that if there is no next number in sequence, that is, if the value of x is 
already MAXINT, then we define the action of Successor to return MAXINT. 
Some programmers might prefer that in such a case Successor return an error 
flag. This is also perfectly permissible. Other functions are Add and Subtract. 
They might also return an error condition, although here we decided to return an 
element of the set NaturalNumber. 0 


Next, we see how the principles of data abstraction and data encapsulation 
heip us to efficiently develop well-designed programs. 


(1) Simplification of software development: The chief advantage of data 
abstraction is that it facilitates the decomposition of the complex task of 
developing a software system into a number of simpler subtasks. Consider the 
following scenario: A problem has been assigned to a single programmer or to a 
team of programmers. Suppose that a top-down review of the problem results in 
a decision that three data types A, B, and C will be used along with some addi- 
tional code (which we will call g/ue) to facilitate interactions among the three 
data types. Assume, also, that the specifications of each data type are provided at 
the outset; i.e., the operations that each data type must support are completely 
specified. 
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ADT NaturaiNumber is 


objects: An ordered subrange of the integers starting at zero and ending at the 
maximum integer (MAXINT) on the computer. 
functions: 
for all x, y € NaturalNumber, TRUE, FALSE € Boolean 
and where +, —, <, ==, and = are the usual integer operations 
Zero(): NaturalNumber 0 
IsZero(x) : Boolean if (x == 0) IsZero = true 
else IsZero = false 
if («+ y <= MAXINT) Addsx+y 
else Add = MAXINT 
Equal(x, y) : Boolean z= if@=y) Equal = TRUE 
else Equal = FALSE 
if (x == MAXINT) Successor = x 
else Successor =x+1 
if (x<y) Subtract =0 
else Subtract =x-y 


Add(x, y) : NaturalNumber = 


Successor(x) : NaturalNumber 


Subtract(x, y) : NaturalNumber 


end NaturalNumber 


ADT 1.1: Abstract data type NaturalNumber 


(a) Scenario 1 - A team of four programmers: the subtasks may be allotted to 
members of a team of four programmers as follows. A programmer is assigned to 
each of the three data types. Each programmer's job is to implement his/her data 
type according to the specifications agreed upon earlier. The fourth programmer 
implements the glue, assuming that the data types are, in fact, implemented 
according to the specifications. No programmer needs to know how the other 
programmers implement their portion of the code. Consequently, each program- 
mer can focus exclusively on his/her portion of the code without worrying about 


what the other programmers are doing and how that might affect what he/she is 
doing. 


(b) Scenario 2 - A single programmer: here data abstraction helps reduce the 
number of things the programmer has to keep in mind at any time. In the exam- 
ple, the programmer would implement each data type one by one according to 
the specifications. When the programmer implements data type A, he/she can do 
so without worrying about data types B and C and the glue. After the individual 
data types are implemented, the programmer can turn his/her attention to the 
glue At this point, the programmer is no longer concemed with the 
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implementations of the three data types, but rather with how to use operations on 
them to achieve the desired goal. 


(2) Testing and Debugging: Breaking down a complex task into a number of 
easier subtasks also simplifies testing and debugging. In the above example, each 
data type can be tested and debugged separately. After each data type is tested 
and debugged, the entire program can be tested. If a bug is detected at this stage, 
it is unlikely that there is a bug in the data structures A, B, and C (represented by 
the shaded boxes of Figure |.1(a)) because they have already been tested. So, the 
bug will most likely be in the glue represented by the unshaded portions of Fig- 
ure 1.1(a). Compare this with the amount of code to be searched if we had nor 
used data abstraction (represented by the unshaded portions Figure 1.1(b)). It is 
easy to see that employing data abstraction in software development leads to 
more efficient testing and debugging. 


Program Code Program Code 


Glue 
(ay by 


Figure 1.1: Unshaded areas represent code that has to be searched for bugs: (a) 
data abstraction is used (b) data abstraction is not used. 


(3) Reusability: Data abstraction and encapsulation typically give rise to data 
structures that are implemented as distinct entities of a software system. This 
makes it easier to extract the code for a data structure and its operations from a 
software system and use it in another software system, rather than if it were inex- 
tricably integrated with the original software system. We will later discuss other 
techniques in C++ that enhance the reusability of software. 


(4) Modifications to the representation of a data type: a significant 
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consequence of information-hiding is that the implementation of a data type is 
not visible to the rest of the program. Specifically, the rest of the program does 
not have direct access to the internal representation of the data type. It manipu- 
lates the data type solely through a suite of operations that the data type has 
made available to it. Thus, a change in the internal implementation of a data type 
will not affect the rest of the program as long as the data type continues to pro- 
vide the same operations, and as long as these operations continue to do the same 
things. All that is required is a change in the implementation of the operations 
that directly access the internal implementation of the data type. 

This is easier to appreciate if we examine the consequences of not employ- 
ing data encapsulation. Consider a program that directly manipulates the internal 
representation of its data types. Suppose a change is made to the implementation 
of a data type. You, the programmer, are now confronted with the tasks of first 
examining the program code to locate instances where the program accesses the 
internal representation of the data type and then making the appropriate changes. 
This could be an extremely laborious task if the program accesses the internal 
representation of the data type many times. 


EXERCISES 


For each of these exercises, provide a definition of the abstract data type using 
the form illustrated in ADT 1.1. 


1. Add the following operations to the NaturalNumber ADT: Predecessor, 
IsGreater, Multiply, Divide. 


2. Create an ADT, Ser. Use the standard mathematical definition and include 
the following operations: Create, Insert, Remove, Isin, Union, Intersec- 
tion, Difference. 

3. 


Create an ADT, Bag. In mathematics a bag is similar to a set except that a 
bag may contain duplicate elements. The minimal operations should 
include Create, Insert, Remove, and isin. 


14 BASICS OF C++ 

The expression “a better C” is often used to describe the C++ language, implying 
that: 

(1) Cand C++ have a lot of features in common. 


(2) C++ has a number of features not associated with either data abstraction or 
inheritance that improve on C. 


In this section we will review features of the C++ language and identify 
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similarities and differences with C. Our presentation assumes that readers are 
familiar with the C language. 


1.4.1 Program Organization in C++ 


As in C, a single C++ program is usually spread out over a number of files. Two 
kinds of files are used in C++: header files and source files. Header files have a 
sh suffix and are used to store declarations. Some header files (such as <ios- 
tream>) are system-defined. (The .h suffix is omitted for system-defined header 
files.) Others are user-defined. Source files are used to store C++ source code. 
The suffix used in source-file names depends on the compiler. We will use a .C 
suffix for source files. Header files are included in the appropriate files through 
the use of the #include preprocessor directive. 

Before a C++ program can run, its source files must be individually com- 
piled, linked, and loaded. Because a C++ program typically consists of many 
source and header files, it is possible that a header file gets included multiple 
times, giving rise to compilation errors. To ensure that a header file is never 
included twice, the contents of a header file are enclosed inside the following 
preprocessor directives. 


#ifndef FILENAME_H 
#define FILENAME_H 
/ insert contents of the header file here 


Hendif 


Another consequence of using multiple source files is that the process of 
typing compilation commands every time we modify and recompile a program 
becomes rather tedious. To avoid this, we suggest that you use the make facility 
in UNIX, or its equivalent, to maintain a C++ program. In addition to saving the 
effort of repeatedly typing compilation commands, make also saves time by not 
recompiling source files that are not affected by modifications made to other files. 


1.4.2 Scope in C++ 


Another consequence of using many files in a single program is that questions 
arise about the visibility of a variable declared in one file and used in other files. 
In view of this, we take a brief look at program scope in C++. C++ program text 
can be classified into one of four scopes: 
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File scope: Declarations that are not contained in a function definition, a class 
definition, or a namespace belongs to a file scope. 


Namespace scope: A namespace is a mechanism that permits logically related 
variables and functions to be grouped together. For example, the standard library 
is defined in a namespace called std. In order to access an entity defined in a 
namespace from outside that namespace, it is necessary to provide the scope 
information. For example, the cout operator in the standard library is accessed 
using std::cout. The repeated use of the scope information may be avoided 
through the use of the using declaration (as in using std). 


Local scope: A name declared in a block belongs to a local scope consisting of 
that block. (A biock is a section of code delimited by a { } pair.) It can also be 
used by sub-blocks contained within the block in which it was declared. 


Class scope: Declarations associated with a class definition belongs to a class 


scope. Each class represents a distinct class scope. We discuss classes in greater 
detail later in Chapter 2. 


Each variable has a scope or a context. A variable is uniquely identified by 
jts scope and its name. A variable is visible to a program only from within its 
scope. For example, a variable defined in a block can only be accessed from 
within the block. A variable defined at file scope (a global variable), however, 
can be accessed anywhere in the program. Some questions that arise as a result 
are 


(4) What do we do if a local variable reuses a global variable name in a block; 
but, we want to access the global variable? 


Solution: use the scope operator :: to access the global variable. 


(2) A global variable is defined in filel.C, but used in file2.C. How do we 
declare the global variable in file2.C? If we declare it, as we normally declare 


variables, in file2.C, the compiler tells us that it is multiply defined. If we don’t 
declare it, the compiler tells us that it hasn't been defined! 
Solution: use extern to declare the variable in file2.C. 


(3) In our program, both filel.C and file2.C define a global variable with the 
same name; but the two global variables are meant to be different entities. How 
do 1 get the program to compile without changing the global variable name in 
one of the files. 


Solution: use static to declare the variables in both files. 


For further details. the reader is referred to one of the introductory texts on C++ 
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listed in the references at the end of the chapter. 


1.4.3 C++ Statements and Operators 


C++ statements have the same syntax and semantics as C statements. C++ state- 
ments are briefly reviewed later in this chapter in the context of performance 
analysis and evaluation. 


C++ operators are identical to C operators with the exception of new and 
delete. We will study new and delete shortly when we discuss dynamic memory 
management in C++. Another difference is that C++ uses the shift left (<<) and 
shift right (>>) operators for input and output. We will also study these shortly 
when discussing input/output in C++. An important difference between C and 
C++ is that C++ allows operator overloading; that is, an operator is allowed to 
have different functions depending on the types of the operands that it is being 
applied to, and furthermore, these functions can be defined by a programmer. We 
will discuss operator overloading in detail in Chapter 2. 


1.4.4 Data Declarations in C++ 


A data declaration associates a data type with a name. We have already listed the 
fundamental types of data provided by C++ in Section 1.3 on Data Abstraction 
and Encapsulation. Now, we list the various options available for declaring data 
in C++ below. All of these options are available in C with the exception of the 
reference type. 


(1) Constant values: these consist of literals such as 5, ’a’, or 4.3. 


(2) Variables: these are instances of data types that can be modified during the 
course of a program. 


(3) Constant variables: these are variables that cannot be assigned a value dur- 
ing their lifetime. Because of this, they must be initialized; that is, their 
contents must be fixed at the time they are declared. A constant type is 
declared by adding the keyword const to its declaration (e.g., const int 
MAX = 5005). 

(4) Enumeration types: This is an alternate mechanism for declaring a series of 
integer constants. It can also be used to create a new data type by giving 
the enumeration type a name: 


enum semester { SUMMER, FALL, SPRING }; 
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SUMMER, FALL and SPRING are constants of type semester. Their 
values are 0, 1 and 2, respectively, by default. 

(5) Pointers: these hold memory addresses of objects. For example, 

int i= 253 

int *np 3 

np = &i; 


Here np is declared to be a pointer to an integer. Next, it is assigned the 


address of integer variable i. So, np is now a pointer to the integer stored in 
variable i (in this case 25). 


(6) Reference types: This is a feature of C++ that is not a feature of C. The 


reference type is a mechanism to provide an alternate name for an object. A 
reference to an object of type T is declared by appending an & to T (i.e., 
T&). For example, 


inti=55 


i=73 " me 
printf ("i= %d, j= %d", i, j) 5 


In the example j is a reference type. It represents an alternate name for i. 
‘When i's value is changed, j’s value also changes correspondingly. When 
the printf statement is executed, the final value of i and j is 7. 


1.4.5 Comments in C++ 


Comments in C++ are specified in two ways: 


(1) Multiline comments: these are identical to comments in C. All text 
enclosed within the /* */ delimiters is ignored by the compiler. 
(2) 


Single line comments: these are unique to C++. All text after // on a line is 
ignored by the compiler. 


The C++ single-line comment format overcomes the following disadavantage of 
the C multi-line comment: if one muli-line comment is nested inside another, the 
portion of the text between the terminating delimiters of the inner and outer com- 
ments is not ignored by the compiler. This could result in the inclusion of code 
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into the program that the programmer believes was commented out. 


1.4.6 Input/Outputin C++ 


To perform V/O in C++, it is necessary to include the system-defined header file 
iostream, The keyword cout is used for output to the standard output device. The 
cout keyword and each entity being printed are separated from each other by the 
<< operator. The entities being output are printed in left to right order on the 
Standard output device. 


#include <iostream> 
main() 
int n = 50 ; float f= 20.3 ; 


cout << "n:" <<cn << endl; 
cout << "f:" << f<< endl ; 


Program 1.1: Output in C++ 


Program 1.1 prints the following on the standard output device: 


n: 50 
f: 20.3 


The cin keyword is used for input in C++. The operator >> is used to separate 
variables being input. Whitespace (i.e., the tab, newline, or blank characters) is 


used to separate items corresponding to different variables on the standard input 
device 


When Program I.2 is executed, both of the following inputs on the standard input 
device result in variable a being set to 5 and b to 10: 


Inputl: 
$ 10 <Enter> 


Input2: 
5 <Enter> 
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#include <iostream> 


main() 

{ 
inta,b5 
cin>>a>>b; 


} 


Program 1.2: Input in C++ 
10 <Enter> 


An advantage of /O in C++ is that it is format-free; that is, the programmer 
is not required to use formatting symbols to specify the type and order of items 
being input/output. Another advantage is that, like other C++ operators, /O 
operators can be overloaded. 

File 1/O in C++ is performed by including the header file fstream as shown 
in Program 1.3. A filestream variable (outFile, in our example) is initialized with 
a string which represents the name of the file (here my.out) and a second argu- 
ment which specifies the mode in which the file will be used (e.g., ios::out 
specifies that the file is to be used for output). If the file is not opened, outFile = 


0. If outFile is successfully opened, it is used instead of cout to direct output to 
file my.out. 


1.4.7 Functions in C++ 


There are two kinds of functions in C++: regular functions and member func- 
tions, Member functions are functions that are associated with specific C++ 
classes. We postpone a detailed discussion of member functions to Chapter 2. 
The features of functions that we review here are common to both types of func- 
tons. 

___ A function consists of a function name, a list of arguments or signature 
(input), a retum type (output), and the body (code that implements a function). 
In Program 1.4, Max is the function name, int a, int 5 is the list of arguments, int 
ss the return type, and the statements between { and ) form the body of the func- 
Hon. 


All functions in C++ return a value. If a function is not meant to retum anything, 
We use void to denote its return type. A value is returned from a function by 
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#include <iostream> 
#include <fstream> 


main(} 


ofstream outFile(“my.out", ios::out); 

if (loutFile) { 
cerr << “cannot open my.out " << endl; // standard error device 
return; 


} 

int n = 50 ; float f= 20.3 ; 
outFile << "n:" <<n<< endl; 
“<< f<<endl; 


J 
Program 1.3: File /O in C++ 


int Max (int a, int b) 

{ 
if (a> b) return a; 
else return 5; 


} 


Program 1.4: An example of a function 


using the return statement. The return statement must retum a value that is of 
the same type as the function's return type or can be converted to the desired 
type. The function terminates when a return statement is encountered. A func- 


tion is invoked by supplying the actual arguments (e.g., Max(x, 100) returns the 
larger of x and 100). 


1.4.8 Parameter Passing in C++ 


In C++, arguments are passed by value. This is the default parameter-passing 
mechanism. When an object is passed by value, it is copied into the function's 
local storage. The function accesses this focal copy. Consequently, any change 
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made to a parameter that is passed by value inside the function only changes its 
local copy. In other words, passing by value does not affect the actual arguments, 

An argument may also be passed by reference. To do this, one needs to 
explicitly declare it to be a reference type. This is done by appending an & to its 
type specifier. In Program 1.4, we can pass a by reference by declaring it as int& 
ain the function’s argument list. When an object is passed by reference, only the 
address of its location is copied into the function’s local store; the object itself is 
not copied into the function's local store. Thus, the function accesses the objects 
referred to by the address; i.e., the actual arguments. 

‘The advantage of passing by value is that variables that were supplied as 
actual arguments to the function are not inadvertently modified by it. The advan- 
tage of passing by reference is that the function executes faster if the object 
being passed requires more memory than its address. This is because the over- 
head of copying the actual argument into the function's local store is replaced by 
the overhead of copying its address. Note that pass-by-reference would be slower 
for an argument type such as char that occupies less memory than its address. 
Another reason for using reference parameters is if you want the function to 
modify the actual argument. This technique is used if a function is to return two 
or more items to the calling function. One of these items is returned by using the 


return statement. The others are returned through the use of reference argu- 
ments. 


One technique for retaining the advantages of both parameter-passing 
methods is to pass constant references such as const T& a, where T is the type of 
the argument. Any attempt to modify a const argument in the function body will 
result in a compile-time error. It is also helpful to use the const keyword to 
characterize an argument that is not modified in a function, because it conveys 
this information to a user of the function. For a more detailed discussion on 
parameter passing mechanisms and the circumstances in which they should be 
used, see the book Effective C++, by Scott Meyers, listed in the references at the 
end of the chapter. 

There is one exception to the rule that the default parameter passing 
mechanism in C++ is pass-by-value: array types are passed by reference; that is, 
the array is nor copied into the function's local store. Therefore, any changes 
made to the array inside the function are reflected in the actual array. When a 
function is invoked with an array argument a (e.g., fla), a pointer to the first 
element of a (i.e.. &a[0}) is actually being passed. This is the reason that, in 
function definitions. arrays are usually denoted by pointers to the appropriate 


type (e.g. flint *a)). Because arrays are passed in this manner, the function does 
hot know the size of the array. This is typically rectified by explicitly passing the 
size of the array as a separate parameter of the function. 
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1.4.9 Function Name Overloading in C++ 


Finally, we discuss function name overloading in C++. Function overloading 
means that there can be more than one function with the same name as long as 
they have different signatures. For example, C++ would allow all the following 
to coexist even though they have the same name Max: 


int Max(int, int); 

int Max(int, int, int); 
int Max(int*, int); 
int Max(float, int); 
int Max(int, float); 


1.4.10 Inline Functions 


An inline function is declared by adding the keyword inline to the function 
definition as follows: 


inline int Sum( int a, int b) 


{ 


returna +6; 


The inline keyword tells the compiler that any calls to Sum must be replaced by 
the body of Sum. This eliminates the overhead of performing a function call and 
copying arguments when the program is executing. So, the statement i = Sum (x, 
12); is replaced by i =x + 12 ;. 

The objective of the inline and const keywords is to eliminate the use of 
preprocessor directives such as #define, which have traditionally been used to 
perform macro substitution. Excessive use of preprocessor directives makes it 
difficult to use programming tools such as debuggers and profilers effectively. 


1.4.11 Dynamic Memory Allocation in C++ 


Objects may be allocated from and released to the free store during runtime 
through the use of the new and delete operators. The new operator creates an 
object of the desired type and returns a pointer to the data type that follows it. If 
it is unable to create the desired object, it throws an exception. An object created 
by new exists for the duration of the program unless it is explicitly deleted by 
applying the delete operator to a pointer to the object: 
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int *ip = new int; 


delete ip ; 


Operators new and delete can also be used to create and delete an array of 
objects as follows: 


int *jp = new int{10) ; // jp is an array of integers 


delete (Lip ; #/ delete the array jp 


The operator ([ ]) is used to inform the compiler that the object being created or 
deleted is an array. 


1.4.12 Exceptions 


Throwing an Exception 

Exceptions are used to signal the occurrence of errors and other special condi- 
tions. For example, the evaluation of the expression a + b *¢ + b/c witha =2, 
b = 1, and c = 0 requires us to divide by zero, which is an error. Although this 
error is not detected by the C++ compiler, your hardware will detect the error 
and throw an exception. 

‘We can write C++ programs that check for exceptional conditions and 
throw an exception when such a condition is detected. For example, we may 
wish to write a function DivZero that computes the expression a + b *c +b/e 
only when each of a, b, and c is greater than 0. Such a function would first check 
that the values of a, b, and c are actually >0. If one or more of these is <0, we 
can signal an exceptional condition by throwing an exception as is done in Pro- 
gram 1.5. The exception thrown by this program is of type char* and the retum 
expression is not evaluated. 

We get more flexibility in processing exceptions when we define an excep- 
tion class (or type) for each of the different kinds of exceptions (e.g., divide by 
zero, illegal parameter value, illegal input value, array index out of range) that 
our program may throw. For example, the C++ operator new that does dynamic 
memory allocation throws an exception of type bad_alloc when it is unable to 
make the requested memory allocation. 

Handling Exceptions 


Exceptions that might be thrown by a piece of code can be handled by enclosing 
this code within a try block. The try block is then followed by zero or more 
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int DivZero(int a, int b, int c) 


if(a <=01!b <=0lIc <=0) 
throw "All parameters should be > 0”; 
returna+b*c+b/c; 


} 


Program 1.5; Throwing an exception of type char* 


catch blocks. Each catch block has a parameter or argument whose type deter- 
mines the type of exception that may be caught by that catch block. For exam- 
ple, the block 


catch (char* e) {} 

catches exceptions of type char* while the block 
catch (bad_ailoc e) {} 

catches exceptions of type bad_alloc. The block 
catch (...) {} 


catches all exceptions regardless of their type. 

A catch block typically contains code to recover from the exception that 
has occurred, or if recovery is not possible, the code in the catch block prints out 
an error message. Program 1.6 shows an example of the try-catch construct. 

Although Program 1.6 has a single catch block following the try block, it 
is possible to follow a try block with several catch blocks. When the code 
within a try block terminates with no exception, we bypass the catch blocks. 
When an exception is thrown, norma! execution of the try block terminates and 
we enter the first catch block that matches an exception of the type thrown. Fol- 
lowing the execution of the code within this matching catch block, we bypass 
the remaining catch blocks. If no catch block matches the thrown exception 
type, then the exception propagates through the hierarchy of nested enclosing try 
blocks to the first catch block in this hierarchy that can handle the exception. If 
the exception is not caught by any catch block, the program terminates abnor- 
mally. 

When Program 1.6 executes, DivZero throws an exception of type char*. 
This exception causes DivZero to terminate without the evaluation of the expres- 
sion. Also, the try block terminates immediately (the cout in the try block 


24 Basic Concepts 


int main() 


try {cout << DivZero (2,0,4) << endl;} 
catch (char? e) 


cout << "The parameters to DivZero were 2, 0, and 4” << endl; 
cout << "An exception has been thrown" << endl; 

cout << e << endl; 

return |; 


return 0; 


} 


Program 1.6: Catching an exception of type char* 


doesn’t complete). Since the type of the exception thrown by DivZero is the 
same as that of the catch block’s parameter ¢, the exception is caught by this 
catch block; ¢ is assigned the thrown exception; and the catch block is entered. 
Figure 1.2 gives the output generated by Program 1.6. 


The parameters to DivZero were 2, 0, and 4 
An exception has been thrown 
All parameters should be > 0 


Figure 1.2: Output from Program 1.6 


EXERCISES 


1. Modify Program 1.5 so that it throws an exception of type int. The value of 


the thrown exception should be 1 if a, b, and ¢ are all less than 0; the value 
should be 2 if all three equal 0. When neither of these conditions is 
satisfied, no exception is thrown. Write a main function that uses your 
modified code, catches the exception if thrown, and outputs a message that 
depends on the value of the thrown exception. Test your code. 


Write a C++ function to retum the sum of the first 2 numbers in the integer 


array a. Your function should throw an exception in case n <0. Test your 
code. 
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1.8 ALGORITHM SPECIFICATION 


1.5.1 Introduction 


The concept of an algorithm is fundamental to computer science. Algorithms 
exist for many common problems, and designing efficient algorithms plays a cru- 
cial role in developing large-scale computer systems. Therefore, before we 
proceed further, we discuss this concept more fully. We begin with a definition. 


Definition: An algorithm is a finite set of instructions that, if followed, accom- 
plishes a particular task. In addition, all algorithms must satisfy the following 
criteria: 


{1) Input. Zero or more quantities are externally supplied. 
(2) Output. At least one quantity is produced. 
(3) Definiteness. Each instruction is clear and unambiguous. 


(4) Finiteness. If we trace out the instructions of an algorithm, then for all 
cases, the algorithm terminates after a finite number of steps. 


(5) Effectiveness. Every instruction must be basic enough to be carried out, in 
principle, by a person using only pencil and paper. It is not enough that 
each operation be definite as in (3); it also must be feasible. O 


In computational theory, one distinguishes between an algorithm and a program, 
the latter of which does not have to satisfy the fourth condition. For example, we 
can think of an operating system that continues in a ‘‘wait’’ loop until more jobs 
are entered. Such a program does not terminate unless the system crashes. Since 
our programs will always terminate, we will use the terms algorithm and pro- 
gram interchangeably in this text. 

We can describe an algorithm in many ways. We can use a natural 
language like English, although if we select this option, we must make sure that 
the resulting instructions are definite. Graphic representations called flowcharts 
are another possibility, but they work well only if the algorithm is small and sim- 
ple. In this text, we will present most of our algorithms in C++, occasionally 
resorting to a combination of English and C++ for our specifications. Two exam- 
ples should help to illustrate the process of translating a problem into an algo- 
rithm. 


Example 1.2 [Selection sort]: Suppose we must devise a program that sorts a 
collection of 1 2 | integers. A simple sofution is given by the following: 
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From those integers that are currently unsorted, find the smallest and 
place it next in the sorted list. 


Although this statement adequately describes the sorting problem, it is not 
an algorithm because it leaves several unanswered questions. For example, it 
does not tell us where and how the integers are initially stored or where we 
should place the result. We assume that the integers are stored in an array, a, 
such that the ith integer is stored in a[i-1}, 1 Si¢n. Program 1.7 is our first 


attempt at deriving a solution. Notice that it is written partially in C++ and par- 
tially in English. 


for (int i= 03 i<n5it+){ 
examine a {i] to a [n—1] and suppose the smallest integer is at a [j]; 
interchange a [i] and a[j} 5 

} 


Program 1.7: Selection sort algorithm 


To tum Program 1.7 into a real C++ program, two clearly defined subtasks 
remain: finding the smallest integer and interchanging it with a[i]. We can solve 
the latter problem by using the code: 


temp = ali); ali] =a{j}, al] = temp; 


We implement this by calling the standard C++ function swap(a[i], ali), 
whose arguments are passed by reference. The first subtask can be solved by 
assuming the minimum is a[i], checking a[i] with ali + 1), a{i +2], «+: and 
whenever a smaller element is found, regarding it as the new minimum. Eventu- 
ally a[1—1) is compared to the current minimum, and we are done. Putting all 
these observations together, we get the function sort (Program 1.8). 


At this point, the obvious question to ask is: Does this function work 
correctly? 


Theorem 1.1: SelectionSort(a,n) correctly sorts a set of n 21 integers; the 
result remains in @[0) -- + a{n—1} such that a(0] Sa[l]<--- sa@[n-1]. 


Proof: We first note that for any i, say i = q, following the execution of lines S- 
9, in is the case that a[g]Sa{r],qg<r<n~t. Also observe that when i 
becomes greater than g, [0] --- a[g] is unchanged. Hence, following the last 
execution of these lines (1e.,i = #—1), we have a[0]<a[l]< --- <a{n-l}. a 


We observe at this point that the upper limit of the for loop in tine 3 can be 
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1 void SelectionSort (int *a, const int n) 
2 {i Sort the n integers a[0] to a[n—1] into nondecreasing order. 


3 for (int i=0;i<n; i++) 

4 { 

5 int j =i; 

6 4 find smallest integer in a[é] to a[n — 1] 
7 for (intk=i+1;k<n; k++) 

8 if (alk) < af) j= ks 

9 swap (ali), a{}); 

10 } 


11} 


Program 1.8: Selection sort 


changed to n — | without damaging the correctness of the algorithm. 


Example 1.3 (Binary search]: Assume that we have n > 1 distinct integers that 
are already sorted and stored in the array a[0] --- a[m—1]. Our task is to deter- 
mine if the integer x is present and if so to retum j such that x = a[j]; otherwise 
retum —1. By making use of the fact that the set is sorted, we conceive of the 
following efficient method: 


Let left and right, respectively, denote the left and right ends of the list to 
be searched. Initially, left = 0 and right = n-1. Let middle = (left+right)/2 be 
the middle position in the list. If we compare a (middle | with x, we obtain one of 
three results: 


{1) x < almiddle]. In this case, if x is present, it must be in the positions 
between 0 and middle ~ 1. Therefore, we set right to middle - 1. 


(2) x ==a[middle]). In this case, we return middle. 


(3) x > afmiddle}. In this case, if x is present, it must be in the positions 
between middle + 1 and n—-1. So, we set left to middle + 1. 


If x has not been found and there are still integers to check, we recalculate mid- 
dle and continue the search. The algorithm contains two subtasks: (1) determin- 
ing if there are any integers left to check and (2) comparing x to almiddle). 


At this point you might try the method out on some sample numbers. This 
method is referred to as binary search. Note how at each stage the number of 
elements in the remaining set is decreased by about one-half. Note also that at 
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each stage, x is compared with a[middle] and depending on whether 
x > a[middle }, x < a(middle}, or x == a(middle ], we do a different thing. We 


can now refine the description of binary search to get a pseudo-C++ function, 
The result is given in Program 1.9. 


int BinarySearch (int *a, const int x, const int n) 
{/ Search the sorted array a {0}, «>> .a{n—1] forx 
Initialize Jeft and right; 
while (there are more elements) 


Let middle be the middle element; 

if (x < a [middle }) set right to middle-1; 
else if (x > a [middle }) set left to middle +1; 
else return middle; 


Not found; 
} 


Program 1.9: Algorithm for binary search 


Another refinement yields the C++ function of Program 1.10, 


int BinarySearch (int *a, const int x, const int n) 
{/ Search the sorted array @ [0], ..., @{n—1] for x. 
int left = 0, right=n -1; 
while (left <= right) 
{// there are more elements 
int middle = (left + right)/2; 
if (& < a[middle }) right = middle-1; 
else if (x > a [middle }) left = middie +1; 
else return middle; 
} ff end of while 


return -1; // not found 
} 


Program 1.10: C++ function for binary search 


To prove this program correct we make assertions about the relati i 
I tionship bi 
Variables before and after each iteration of the while loop. of this loop 


As we enter this loop 
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and if x is present in a, the following holds: 
left S right && alleft]<x < alright] && SORTED (a, n) 


Now, if control passes out of the while loop, then we know the condition of 
while loop is false, so left > right. This, combined with the above assertion, 
implies that x is not present. 

Unfortunately, a complete proof takes us beyond the scope of this text, but 
those who wish to pursue program-proving should consult the references at the 
end of this chapter. 0 


1.5.2 Recursive Algorithms 


We have emphasized the need to structure a program to make it easier to achieve 
the goals of readability and correctness. One of the most useful syntactical 
features for accomplishing this is the function. A set of instructions that perform 
a logical operation, perhaps a very complex and long operation, can be grouped 
together as a function. The function name and its parameters are viewed as a 
new instruction that can be used in other programs. Given the input-output 
specifications of a function, we do not even have to know how the task is accom- 
plished, only that it is available. This view of the function implies that it is 
invoked, executed and returns control to the appropriate place in the calling 
function. What this fails to stress is that functions may call themselves (direct 
recursion) before they are done or they may call other functions that again 
invoke the calling function (indirect recursion). These recursive mechanisms 
are extremely powerful, but even more importantly, often they can express an 
otherwise complex process very clearly. For these reasons we introduce recur- 
sion here. 

Recursion is similar to the method of induction which is often used to 
prove mathematical statements. In mathematical induction, a statement about 
integers (e.g., the sum of the first n positive integers is n(#+1)/2) is proved by 
showing that the statement can be proved for integer & if it is assumed to be true 
for integer k - 1. Similarly, in recursion, we write a function to produce an output 
(say n!) for some input (here, n) by assuming that the same function will compute 
the correct output for input 2-1. In mathematical induction, we need a basis 
which can be directly proved (that is, the proof for the basis does not make any 
assumptions). Similarly, a recursive function requires a terminating condition. 
When the input to the function satisfies this terminating condition, the function 
directly computes the output without calling itself. 

What kinds of problems are best solved by recursion? Typically, beginning 
programmers view recursion as a somewhat mystical technique that is useful 
only for some very special class of problems (such as computing factorials or 
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Ackermann’s function). This is unfortunate because any program that can be 
written using assignment, the if-else statement, and the while statement can also 
be written using assignment, if-else, and recursion. Of course, this does not 
mean that the resulting program will necessarily be easier to understand. How- 
ever, there are many instances when this will be the case. When is recursion an 
appropriate mechanism for algorithm exposition? One instance is when the 


problem itself is recursively defined. Factorial fits this category, as well as bino- 
mial coefficients where 


{ml ee oe al 


can be recursively computed by the formula 


n) _ [n-1 n-1 
(n) = (*m') + (a) 
‘We use two examples to show you how to develop a recursive algorithm. In 
the first example, we take the binary search function that we created in Example 


1.3 and transform it into a recursive function. In the second example, we gen- 


erate all possible permutations of a list of characters. To understand a recursive 
function, you must 


(1) Formulate in your mind a statement of what it is that the function is sup- 
posed to do, for a given input. 


(2) Verify that the function does achieve its goal if the recursive invocations to 
itself do what they are supposed to. 

(3) Ensure that a finite number of recursive invocations of the function eventu- 
ally lead to an invocation which satisfies the terminating condition (other- 
wise, the function will keep calling itself and not terminate!). 


(4) The function should perform the correct computations if the terminating 
condition is encountered. 


Example 1.4 [Recursive binary search]; Program 1.10 gave the iterative version 
of binary search. In the recursive version we pass left and right as parameters 
(Program 1.11). The for loop of Program 1.10 has been replaced by recursive 
calls in Program 1.11. To invoke the recursive function, we use the statement 


BinarySearch(a, x,0,n—-1); 


You should verify that BinarySearch satisfies the four conditions stated above for 
recursive functions. Notice that both the iterative (Program 1,10) and recursive 
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(Program 1.11) functions perform the same computation. 0 


int BinarySearch (int *a, const int x, const int left, const int right) 
{/ Search the sorted array a[left], ---, afright] for x 
if (left <= right) { 
int middle = (left + right)/2; 
if (x < a[middle }) return BinarySearch(a, x, left, middle — 1); 
else if (x > a [middle ]) return BinarySearch(a, x,middle + 1, right); 
return middle; 
} end of if 
return -1; // not found 


} 


Program 1.11: Recursive implementation of binary search 


Example 1.5 [Permutation generator}: Given a set of n 2 1 elements, the prob- 
lem is to print all possible permutations of this set. For example if the set is 
{a, b, c}, then the set of permutations is {(a, b, c), (a, ¢, b), (b, a, ¢), (b, ¢, a), 
(c, a, b), (c, b, @)}. It is easy to see that given n elements, there are ! different 
permutations. A simple algorithm can be obtained by looking at the the case of 
four elements (a,b,c,d). The answer can be constructed by writing 


(1) a followed by all permutations of (b,c,d) 
(2) b followed by all permutations of (a,c,d) 
(3) _¢ followed by all permutations of (a,b,d) 
(4) d followed by all permutations of (¢,b,c) 


The expression ‘followed by all permutations”’ is the clue to recursion. It 
implies that we can solve the problem for a set with n elements if we have an 
algorithm that works on n- 1 elements. These observations lead to Program 
1.12, which is invoked by Permutations (a, 0,n — 1). 

Try this algorithm out on sets of length one, two, and three to ensure that 
you understand how it works. 0 


Another time when recursion is useful is when the data structure that the 
algorithm is to operate on is recursively defined. We shall see several important 
examples of such structures in this book. 
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void Permutations (char *a, const int k, const int m) 
{// Generate all the permutations of a[k], +++ ,a{m]. 
if (k ==) { // output permutation 
for (int i = 0; i <= m; i++) cout << ali] << "5 
cout << endl; 


else // a[k:m] has more than one permutation. Generate these recursively. 
for (i = k; i <= mj i++) { 
swap (a{k], ali}); 
Permutations (a, k + 1, m); 
swap (a{k}, alé]); 


} 


Program 1.12: Recursive permutation generator 


EXERCISES 


1. Homer's rule is a means for evaluating a polynomial A(x) = 
aX" + yx"! + ++ > + \X + Qo at a point xo using a minimum number 
of multiplications. This rule is: 

AQ) = (+> @yX0 + Gyo + ++ +21) XQ + ag 


Write a C++ program to evaluate a polynomial using Homer’s rule. Deter- 
mine how many times each statement is executed. 

2. Given n Boolean variables x,, «+, x, we wish to print all possible combi- 
nations of truth values they can assume. For instance, if n = 2, there are 
four possibilities: true, true; true, false; false, true; false, false. Write a 
C++ program to accomplish this and do a frequency count. 

3. Trace the action of the code 


i=O;jen-1y 

do{ 
kai + jy2; 
if@[k)<=x)i=k+1; 
else j =k ~ 15} 


while (i <= j); 


on the elements 2, 4, 6, 8, 10, 12, 14, 16, 18, and 20 searching for x = 1, 3, 
13, of 21. 


10. 


14. 
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Write a C++ program that prints out the integer values of x, y, and z in non- 
decreasing order. What is the computing time of your method? 

Write a C++ function that searches an unsorted array a[0:n~1] for the ele- 
ment x. If x occurs, then return the leftmost position of x in the array, else 
return —1. 

The factorial function n! has value 1 when ns 1 and value n*(n—1)! when 
n> 1, Write both a recursive and an iterative C++ function to compute n!. 
The Fibonacci numbers are defined as: fg = G, fy = 1, and f = fi-1 +f)-2 for 
i> 1, Write both a recursive and an iterative C++ function to compute fj. 
Write a recursive function for computing the binomial coefficient is as 
defined in Section 1.5.2, where (0] = (?] = 1, Analyze the time and 
space requirements of your algorithm. 


Write an iterative function to compute a binomial coefficient; then 
transform it into an equivalent recursive function. 


Ackermann’s function A (m,n) is defined as follows: 


n+l ,ifm=0 
A(mn) = {A(m-1, 1) »ifn=0 
A(m — 1, A(mn — 1)) , otherwise 


This function is studied because it grows very fast for small values of 
and n. Write a recursive function for computing this function. Then write 
a nonrecursive algorithm for computing Ackermann’s function. 
The pigeonhole principle states that if a function f has n distinct inputs but 
less than n distinct outputs, then there exist two inputs @ and b such that 
a#band f(a) = f(b). Write a program to find the values a and b for which 
the range values are equal. Assume that the inputs are 1, 2, ---, 7. 
Given n, a positive integer, determine if n is the sum of all of its divisors — 
i.e., if 2 is the sum of all ¢ such that 1 $f < a, and ¢ divides n. 
Consider the function F (x) defined by 

if (xis even) F =x/2; 

else F = F(F (3x + 1); 
Prove that F(x) terminates for all integers x. (Hint: Consider integers of 
the form (2: + 1)2* — } and use induction.) 
Tf Sis a set of n elements, the powerset of S is the set of all possible subsets 


of S. For example, if S = (a,b,c), then powerset (S) = {(), (a). (b). (c), 
(a,b), (a,c), (b,c), (a,b,c)}. Write a recursive function to compute 
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powerset (S). 

15. {Towers of Hanoi] There are three towers and sixty-four disks of different 
diameters placed on the first tower. The disks are in order of decreasing 
diameter as one scans up the tower. Monks were supposed to move the 
disks from tower | to tower 3 obeying the following rules: (a) only one 
disk can be moved at any time and (b) no disk can be placed on top of a 
disk with smaller diameter. Write a recursive function that prints the 
sequence of moves that accomplish this task. 


16 THE STANDARD TEMPLATE LIBRARY 


The C++ standard templates library (STL) is a collection of containers, adaptors, 
iterators, function objects (also known as functors) and algorithms. Through the 
judicious use of elements of the STL, the task of writing application codes is 
greatly simplified. In this section, we introduce a few of these elements. To use 
these STL elements in your programs, you should add the statement 


#include <algorithm> 


to your programs. 


Example 1.6 [The STL algorithm accumulate}: The STL has an algorithm 
accumulate that may be used to sum the elements in a sequence, The syntax is 


accumulate (start, end, initialValue) 


where start points to the first element to be accumulated and end points to one 
position after the last element to be accumulated. So, elements in the range 
{start, end) are accumulated. The invocation 


accumulate (a, a +n, initialValue) 


where a is a one-dimensional array, for example, returns the value 


not 
initialValue + Sati] 
i=0 


The STL algorithm accumulate accesses successive elements of the 
Sequence to be summed by performing the ++ operator on start and terminating 
when the pointer value becomes end So, this algorithm may be used to sum the 
values of any sequence whose elements may be obtained by repeated application 
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of the ++ operator. One-dimensional arrays and the STL container vector are 
two examples of sequences whose elements may be accessed in this way. We 
shall see other examples later in this book. 

The STL has a more general form of the accumulate algorithm, which has 
the following syntax 


accumulate (start, end, initialValue, operator) 


where operator is a function that defines the operation to be used during the 
accumulation process. Using the STL functor multiplies<int>, which multiplies 
two integers, we can find the product of an array of integers using the code of 
Program 1.13. O 


int Product(int *a , int n) 
{// Return sum of the numbers a [0:2~1]. 
int initVal = 1; 
return accumulate (a, a +n, initVal, multiplies<int>()); 


} 


Program 1.13: Compute the product of the elements a (0:n-1] 


Example 1.7: [The STL algorithms copy and next_permutation] The copy algo- 
rithm copies a sequence of elements from one location to another. The syntax is 


copy (start, end, to) 


where fo gives the location to which the first element is to be copied. So, ele- 
ments are copied from locations start, start + 1, «-+, end — 1} to the locations fo, 
to +1, +-+-,to + end — start, 

The algorithm next_permutation, which has the syntax 


next_permutation (start, end) 


creates the next lexicographically larger permutation of the elements in the range 
(start, end); it returns the value true if-and-only-if (iff) such a next permutation 
exists. By starting with the smallest lexicographic permutation of a sequence of 
distinct elements and making successive calls to next_permutation, we can 
obtain all permutations. Program 1.14 does just this. The invocation of copy in 
this program copies the elements a [0:m] to the output stream cout; each copied 
element is followed by a space (" "). Notice that Program 1.14 outputs no 
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permutation that is lexically smaller than the initial sequence whereas Program 
1.15 outputs all permutations regardless of the initial sequence. The exercises 
examine ways to modify Program 1.14 so as to obtain all permutations, 


void Permutations(char *a, const int m) 
{// Generate all permutations of a[0:m }. 


} 


#/ Output the permutations one by one 
do{ 


copy (a, a + m + 1, ostream_iterator<char>(cout, 
cout << endl; 


} while (next_permutation(a,a + m + 1)); 


Program 1.14: Permutations using the STL algorithm next_permutation 


A more general form of the next_permutation algorithm takes a third 


parameter compare as in 


next_permutation (start, end, compare) 


When this form is used, the binary function compare is used to determine 


whether one element is smaller than another. In the two-parameter version, this 
comparison is done using the operator <. 0 


The STL contains many algorithms in addition to the ones used in the 


preceding two examples. The exercises explore STL algorithms further. 


EXERCISES 


Modify Program 1.14 so that it outputs all permutations of distinct ele- 
ments. Do this by sorting the element list into ascending order prior to gen- 
erating the permutations. To sort, use the STL algorithm 


sort (start, end) 


which sorts elements in the range [start, end) into ascending order. Test 
your code. 

Modify Program 1.14 so that it outputs all permutations of distinct ele- 
ments, Do this by first using next_permutation to generate permutations 
that are lexically larger than the initial permutation and then using the STL 


1.7 
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algorithm prev_permutation to generate permutations that are lexically 
smaller than the initial permutation. Test your code. 

Modify Program 1.14 so that it outputs all permutations of distinct ele- 
ments, Do this by using the fact that when next_permutation retums the 
value false the sequence [start, end) is the lexically smallest sequence. 
Hence, subsequent invocations of next_permutation will get you the 
remaining (if any) permutations you need. Test your code. 


The STL algorithm count, which has the syntax 

count (start, end, value) 
returns the number of occurrences of value in the range [start, end). Write 
a program that uses this algorithm to determine the number of occurrences 
of a [0] in the integer array a [0:n—1]. Test your code. 
The STL algorithm fill which has the syntax 

fill(start, end, value) 

sets all positions in the range [start, end) to value. Write a program that 


creates an integer array of a specified size and initializes all positions of 
this array to 0. Test your code. 


PERFORMANCE ANALYSIS AND MEASUREMENT 


One goal of this book is to develop skills for making evaluative judgments about 
programs. There are many criteria upon which we can judge a program, for 


instance: 

(1) Does it do what we want it to do? 

(2) Does it work correctly according to the original specifications of the task? 

(3) _ Is there documentation that describes how to use it and how it works? 

(4) Are functions created in such a way that they perform logical subfunc- 
tions? 

(5S) Is the code readable? 


The above criterta are all vitally important when it comes to writing software, 
most especially for large systems. Though we will not be discussing how to 
teach these goals, we will try to achieve them throughout this book with the pro- 
grams we write. Hopefully this more subtle approach will gradually infect your 
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own program-writing habits so that you will automatically strive to achieve these 
goals. 


‘There are other criteria for judging programs that have a more direct rela- 


tionship to performance. These have to do with their computing time and storage 
Tequirements. 


Definition: The space complexity of a program is the amount of memory it 
needs to run to completion. The time complexity of a program is the amount of 
computer time it needs to run to completion. 0 


Performance evaluation can be loosely divided into two major phases: (1) 
a priori estimates and (2) a posteriori testing. We refer to these as performance 
analysis and performance measurement respectively. 


1.7.1 Performance Analysis 


1.7.1.1 Space Complexity 

Function Abc (Program 1.16) computes the expression a+b tbc + 

(a+b-c)/(a +b)+4.0; function Sim (Program 1.17) computes the sum > a[i] 
i=0 

iteratively, where the a[i]’s are of type float; and function Rsum (Program 1.18) 


is a recursive program that computes > a[i]. 
i=0 


float Abc(float a, float b, float c) 
{ 


return a+b +b *c +(a +b-c Ma +b)+4.0; 
} 


Program 1.16: Function to compute a +b +b*c + (a +b-c Va +b)+4.0 


The space needed by each of these programs is seen to be the sum of the 
following components: 


(1) _ A fixed part that is independent of the characteristics (¢.g., number, size) of 
the inputs and outputs. This part typically includes the instruction space 
(ie, space for the code), space for simple variables and fixed-size com- 
ponent variables (also called aggregate), space for constants, etc. 
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line float Sun (float *a, const int n) 
If 

2 float s = 0; 

3 for (int i = 0; i <n; i++) 

4 st=ali); 

5 return 5; 

6 


} 


Program 1.17; Iterative function for sum 


line float Rsum (float +a, const int n) 


1 

2 if (n <= 0) return 0; 

3 else return (Rsum(a, n-1) + a[n—1}); 
4} 


Program 1.18: Recursive function for sum 


(2) A variable part that consists of the space needed by component variables 
whose size is dependent on the particular problem instance being solved, 
the space needed by referenced variables (to the extent that this depends on 
instance characteristics), and the recursion stack space (insofar as this 
space depends on the instance characteristics). 


The space requirement S(P) of any program P may therefore be written as $(P) 
=c + Sp(instance characteristics) where c is a constant. 

When analyzing the space complexity of a program, we shall concentrate 
solely on estimating Sp (instance characteristics). For any given problem, we 
shall need to first determine which instance characteristics to use to measure the 
space requirements. This is very problem-specific, and we shall resort to exam- 
ples to illustrate the various possibilities. Generally speaking, our choices are 
limited to quantities related to the number and magnitude of the inputs to and 
outputs from the program. At times, more complex measures of the interrelation- 
ships among the data items are used. 


Example 1.8: For Program 1.16, the problem instance is characterized by the 
specific values of a, b, and c. Making the assumption that one word is adequate 
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to store the values of each of a, b, c, and the value retuned by Abc, we see that 
the space needed by function Abc is independent of the instance characteristics. 
Consequently, Sp(instance characteristics) = 0. 0 


Example 1.9: The problem instances for Program 1.17 are characterized by 7, 
the number of elements to be summed. Since n is passed by value, one word 
must be allocated for it. Since a is actually the address of the first element in a[] 
(ie. a [O}), the space needed by it is also one word. So, the space needed by the 
function is independent of n and S gym(n) = 0.0 


Example 1.10: Let us consider the function Rsum. As in the case of Sum, the 
instances are characterized by x. The recursion stack space includes space for 
the formal parameters, the local variables, and the return address. Since a is the 
address of a0}, it requires only one word of memory on the stack. Assume that 
the retum address requires only one word of memory. Each call to Rsum 
requires at least 4 words (including space for the values of n, a, the returned 
value, and the return address). Since the depth of recursion is n +1, the recursion 
stack space needed is 4(n +1). For n = 1000, the recursion stack space is 4004. 0 


1.7.1.2 Time Complexity 


The time, T(P), taken by a program P is the sum of the compile time and the run 
(or execution) time. The compile time does not depend on the instance charac- 
teristics. Also, we may assume that a compiled program will be run several 
times without recompilation. Consequently, we shall concern ourselves with just 
the run time of a program. This run time is denoted by tp {instance characteris- 
tics), 

Because many of the factors tp depends on are not known at the time a pro- 
gram is conceived, it is reasonable to attempt only to estimate tp. If we knew the 
characteristics of the compiler to be used, we could proceed to determine the 
number of additions, subtractions, multiplications, divisions, compares, loads, 
stores, and so on that would be made by the code for P. So, we could obtain an 
expression for tp(n) of the form 


tp(n) = cgADD(n) + c,SUB(n) + ¢,,MUL(n) + cgDIV(n) + ++ - 


where x denotes the instance characteristics, and cq, Css Cm Cg, ClC., respectively 
denote the time needed for an addition, subtraction, multiplication, division, etc., 
and ADD, SUB, MUL, DIV, etc., are functions whose value is the number of 
additions, subtractions, multiplications, divisions, etc., that will be performed 
when the code for P is used on an instance with characteristic 7. 

Obtaining such an exact formula is in itself an impossible task, since the 
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time needed for an addition, subtraction, multiplication, etc., often depends on 
the actual numbers being added, subtracted, multiplied, etc. In reality then, the 
true value of tp() for any given n can be obtained only experimentally. The 
program is typed, compiled, and run on a particular machine. The execution 
time is physically clocked and tp(n) obtained. Even with this experimental 
approach, one could face difficulties. In a multiuser system, the execution time 
will depend on such factors as system load, the number of other programs run- 
ning on the computer at the time program P is run, the characteristics of these 
other programs, and so on. 

Given the minimal utility of determining the exact number of additions, 
subtractions, etc., that are needed to solve a problem instance with characteris~- 
tics given by n, we might as welt lump all the operations together (provided that 
the time required by each is relatively independent of the instance characteris- 
tics) and obtain a count for the total number of operations. We can go one step 
further and count only the number of program steps. 

A program step is loosely defined as a syntactically or semantically mean- 
ingful segment of a program that has an execution time that is independent of the 
instance characteristics. For example, the entire statement 


returna+b+b*c¢+(a+b~c)/(a+b)+ 4.0; 


of Program 1.16 could be regarded as a step since its execution time is indepen- 
dent of the instance characteristics (this statement is not strictly true, since the 
time for a multiply and divide will generally depend on the actual numbers 
involved in the operation). 

The number of steps any program statement is to be assigned depends on 
the nature of that statement. The following discussion considers the various 
statement types that can appear in a C++ program and states the complexity of 
each in terms of the number of steps: 


(1) Comments. Comments are not executable statements and have a step count 
of zero. 


(2) Declarative statements. This category includes all statements that define 
or characterize variables and constants (int, long, short, char, float, dou- 
ble, const, enum, signed, unsigned, static, extern), all statements that 
enable users to define their own data types (class, struct, union, template), 
all statements that determine access (private, public, Protected, friend), 
and all statements that characterize functions (void, virtual). These count 
as zero steps, since these are either not executable or their cost may be 
Jumped into the cost of invoking the function they are associated with. 


(3) Expressions and assignment statements. Most expressions have a step 
count of one. The exceptions are expressions that contain function calls. 


42 Basic Concepts 


(4) 


(5) 


In this case, we need to determine the cost of invoking the functions. This 
cost can be large if the functions employ many-element pass-by-value 
parameters because the values of all actual parameters need to be assigned 
to the formal parameters, This is discussed further under function and 
function invocation. When the expression contains functions, the step 
count is the sum of the step counts assignable to each function invocation. 
The assignment statement <variable> = <expr> has a step count equal 
to that of <expr> unless the size of <variable> is a function of the instance 
characteristics. In this latter case, the step count is the size of <variable> 
plus the step count of <expr>. For example, the assignment a = b, where @ 


and b are of type ElementList, has a step count equal to the size of 
ElementList. 


lteration statements. This class of statements includes the for, while, and 
do statements. We shall consider the step counts only for the control part 
of these statements. These have the following form: 


for (<init-stmt>; <exprl>; <expr2>) 
while <expr> do 
do ...while <expr> 


Each execution of the control part of a while and do statement will be 
given a step count equal to the number of step counts assignable to <expr>. 
The step count for each execution of the control part of a for statement is 
one, unless the counts attributable to <init-stmt>, <exprl>, or <expr2> are 
a function of the instance characteristics. In this latter case, the first execu- 
tion of the control part of the for has a step count equal to the sum of the 
counts for <init-stmt> and <expr1>; subsequent executions of the for state- 
ment have a step count equal to the sum of the step counts for <expri> and 
<expr2>. 
Switch statement. This statement consists of a header followed by one or 
more sets of condition-statement pairs. We shail once again consider the 
costs for the control part of the statement. 


switch (<expr>) { 
case cond!: <statementl> 
case cond2: <statement2> 


default: <statement> 


(6) 


7) 


(8) 


(9) 
(10) 
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The header switch (<expr>) is given a cost equal to that assignable to 
<expr>. The cost of each condition is its cost plus that of all preceding 
conditions. 


If-else statement. The if-else statement consists of three parts: 


if (<expr>) <statements1>; 
else <statements2>; 


Each part is assigned the number of steps corresponding to <expr>, <state- 
mentsi>, and <statements2> respectively. Note that if the else clause is 
absent, then no cost is assigned to it. Step counts are computed for the 
arithmetic-if operation in a similar manner. 


Function invocation. Ail invocations of functions count as one step unless 
the invocation involves pass-by-value parameters whose size depends on 
the instance characteristics. In this latter case, the count is the sum of the 
sizes of these value parameters. If the function being invoked is recursive, 
then we must also consider the local variables in the function being 
invoked. The sizes of local variables that are characteristic-dependent 
need to be added to the step count. 


Memory management statements. These include new object, delete object, 
and sizeof(object). The step count associated with each is 1. The state- 
ments new and delete can potentially invoke the constructor and destructor 
for object, respectively. In this case, the step counts are computed in a 
similar manner to a function invocation. 


Function statements. These count as zero steps because their cost has 
already been assigned to the invoking statements. 


Jump statements. These include continue, break, goto, return, and 
return <expr>. Each has a step count of one, with the possible exception 
of return <expr>. This has a step count of 1 unless the step count attribut- 
able to <expr> is a function of instance characteristics. In this case, the step 
count is the cost of <expr>. 


With the above assignment of step counts to statements, we can proceed to 


determine the number of steps needed by a program to solve a particular problem 
instance. We can go about this in one of two ways. In the first method, we intro- 
duce a new variable, count, into the program. This is a global variable with ini- 
tial value 0. Statements to increment count by the appropriate amount are intro- 
duced into the program. This is done so that each time a statement in the original 
program is executed, count is incremented by the step count of that statement. 


Example 1.11: When the statements to increment count are introduced into Pro- 
gram 1,17, the result is Program 1.19. The change in the value of counr by the 


44 Basic Concepts 


time this program terminates is the number of steps executed by Program 1.17. 
Since we are interested in determining only the change in the value of 
count, Program 1.19 may be simplified to Program 1.20. It should be easy to see 
that for every initial value of count, Program 1.19 and Program 1.20 compute the 
same final value for count. It is easy to see that in the for loop, the value of 
count will increase by a total of 2n. If count is zero to start with, then it will be 


2n+3 on termination. So, each invocation of Sum (Program 1.17) executes a 
total of 2n +3 steps. O 


float Sum (float +a, const int n) 
{ 
float s = 0; 
count++; Hf count is global 
for (int i = 0; i <n; i++) { 
count++5 / for for 
s+=ali); 
count++; // for assignment 


count++; / for last time of for 
count++; // for return 
return s; 


} 


Program 1.19; Program 1.17 with count statements added 


void Sum (float *a, const int n) 


for (int i=0; i <n; i++) 
count += 2; 
count += 3; 


Program 1.20: Simplified version of Program 1.19 


Example 1.12: When the statements to increment count are introduced into Pro- 
gram 1.18, Program 1.21 is obtained. Let tgsun(”) be the increase in the value of 
count when Program 1.21 terminates. We see that trsum(0) = 2. When n > 0, 
count increases by 2 plus whatever increase results from the invocation of Ruan 
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from within the else clause. From the definition of ¢pgum, it follows that this addi- 
tional increase is trym(n—1). So, if the value of count is zero initially, its value 
at the time of termination is 2+fpyn(n—1), 2 > 0. 


float Rsum (float *a, const int 2) 


{ 
count++; I! for if conditional 
if (2 <=0) { 
count++; /f for return 
return 0; 
} 
else { 
count++ ; // for return 
return (Rsum(a,n — 1) + a[n— 1)); 
} 


Program 1,21: Program 1.18 with count statements added 


* When analyzing a recursive program for its step count, we often obtain a 
recursive formula for the step count (i.€., SAY tReum(t) = 2+fpum(n—L), 2 >0 and 
tRsum(O) = 2). These recursive formulas are referred to as recurrence relations. 
This recurrence may be solved by repeatedly substituting fr tren as below: 


"Rsum(2) = 2+ tesum(n—-1) 
=242+ teeum(t-2) 
22X24 tRsum(n-2) 


= 20 + tpum(0) 
=2n+2 


So, the step count for function Rsum (Program 1.18) is 2n +2.0 


Comparing the step count of Program 1.17 to that of Program 1.18, we see 
that the count for Program 1.18 is less than that for Program 1.17. From this, we 
cannot conclude that Program 1.17 is slower than Program 1.18. This is so 
because a step does not correspond to a definite time unit. Each step of Rsum 
may take more time than every step of Sum. So, it might well be (and we expect) 
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that Rsum is slower than Sum. 


The step count is useful in that it tells us how the run time for a program 
changes with changes in the instance characteristics. From the step count for 
Sum, we see that if n is doubled, the run time will also double (approximately); if 
n increases by a factor of 10, we expect the run time to increase by a factor of 10; 
and so on. So, we expect the run time to grow linearly in n. 


Example 1.13 [Matrix addition]: Program 1.22 is a program to add two m Xn 
matrices a and b together. Note that the argument **@ refers to a two- 
dimensional array @(][J. Introducing the count-incrementing statements leads to 
Program 1.23. Program 1.24 is a simplified version of Program 1.23 that com- 
putes the same value for count. Examining Program 1.24, we see that line 5 is 
executed n times for each value of i or a total of mn times; line 6 is executed m 
times; and line 8 is executed once. If count is zero to begin with, it will be 
2mn +2m +1 when Program 1.24 terminates. 

From this analysis we see that if m>n, then it is better to interchange the 
two for statements in Program 1.22. If this is done, the step count becomes 
2mn+2n+1. Note that in this example the instance characteristics are given by 


manda. O 

line void Add (int **a, int **b, int **c, int m, int n) 
1 eats th 
2° for (inti= Osi < mi; i++) 


3 for (int j=0 3 j <n 5 j++) 
; } eff)U) = aff) + 6 


Program 1.22: Matrix addition 


The second method to determine the step count of a program is to build a 
table in which we list the total number of steps contributed by each statement. 
This figure is often arrived at by first determining the number of steps per execu- 
tion of the statement and the total number of times (i.e., frequency) each state- 
ment is executed. By combining these two quantities, the total contribution of 
each statement is obtained. By adding up the contributions of all statements, the 
step count for the entire program is obtained. 

There is an important difference between the step count of a statement and 
its steps per execution (s/e). The step count does not necessarily reflect the com- 
plexity of the statement. For example, the statement 


x= Sum(a, m); 
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line void Add (int **a, int +*b, int **c, int m, int n) 


for (int i = 0; i < m ; i++) 
i 
count++; Hf for for i 
for (int j = 03 j <n; j++) 


count++3 Hf for for j 

cf) = 2lU) + bf): 

count++; // for assignment 
count++; // for last time of for j 


count++; // for last time of for i 


Program 1.23: Matrix addition with counting statements 


line void Add (int **a, int **b, int **c, int m, int n) 


1 

2 for (int i= 05 i < m; i++) 
3 

4 for (int j = 0; j <n 3 j++) 
5 count += 2; 

6 count += 2; 

7 } 

8 count++; 

9 


} 


Program 1.24: Simplified program with counting only 


has a step count of 1, while the total change in count resulting from the execution 
of this statement is actually 1 plus the change resulting from the invocation of 
Sum (i.¢., 2m +3). The steps per execution of the above statement is 1+2m+3 = 
2m+4. The s/e of a statement is the amount by which count changes as a result 
of the execution of that statement. 

In Table 1.1, the number of steps per execution and the frequency of each 
of the statements in function Sum (Program 1.17) have been listed. The total 
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number of steps required by the program is determined to be 2n +3. It is impor- 
tant to note that the frequency of line 3 is n+1 and not n. This is so because i 
has to be incremented to 1 before the for loop can terminate. 


line sle frequency _ total steps 


1 0 1 i) 

2 1 1 1 

3 1 ntl navi 

4 1 nr n 

5 1 1 1 

6 0 1 0 
Total number of steps 2n+3 


Table 1.1: Step table for Program 1.17 


Table 1.2 gives the step count for function Rsum (Program 1.18). Line 2(a) 
refers to the if conditional of line 2, and line 2(b) refers to the statement in the if 
clause. Notice that under the s/e (steps per execution) column, line 3 has been 
given a count of 1+tpyum(—1). This is the total cost of line 3 each time it is exe- 
cuted. It includes all the steps that get executed as a result of the invocation of 
Rsum from line 3. The frequency and total steps columns have been split into 
two parts: one for the case n = 0 and the other for the case 1 >0. This is neces- 
sary because the frequency (and hence total steps) for some statements is 
different for each of these cases. 


line s/e frequency total steps 
n=0 n>0 n=0  n>O0 

1 0 1 1 0 0 
2a) 1 ,] 1 1 1 
20) 1 1 0 1 0 
3 14+tRsun(a—1) 0 1 i) 1+fRsun(n~1) 
4 0 1 1 0 it) 

Total number of steps 2 2+trsumt—1) 


Table 1.2: Step table for Program 1.18 


Table 1.3 corresponds to function Add (Program 1.22). Once again, note 
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that the frequency of line 2 is m+1 and not m. This is so as i needs to be incre- 
mented up to m before the loop can terminate. Similarly, the frequency for line 3 
is m(+1). When you have obtained sufficient experience in computing step 
counts, you may avoid constructing the frequency table and obtain the step count 
as in the following example. 


line sle frequency total steps 
1 0 1 0 

2 1 m+ m+) 

3 1 m(n+1) matm 

4 1 mn mn 

- 0 1 tt) 


Total number of steps 2mn+2m+1 


Table 1.3: Step table for Program 1.22 


Example 1.14 [Fibonnaci numbers): The Fibonacci sequence of numbers starts 
as 


0, 1, 1, 2,3, 5, 8, 13, 21, 34, 55, --- 


Each new term is obtained by taking the sum of the two previous terms. If we 
call the first term of the sequence Fy then Fy = 0, F, = 1, and in general 


F,, = F,-) + Fy-2, 22. 


The program Fibonacci (Program 1.25) inputs any nonnegative integer n and 
prints the value F,,. 

To analyze the time complexity of this program, we need to consider the 
two cases: (t) n = 0 or 1 and (2) n > I. Line 3 will be regarded as two lines: 
3(a), the conditional part, and 3(b), the if clause. When x = 0 or 1, lines 3(a) and 
3(b), get executed once each. Since each line has an s/e of.1, the total step count 
for this case is 2. When n > 1, lines 3(a), 5, and 12 are each executed once. Line 
6 gets executed # times, and lines 7-11 get executed n~1 times each (note that 
the last time line 6 is executed, i is incremented to n + 1 and the loop exited). 
Line 5 has an s/e of 2; the remaining lines that get executed have an s/e of I. The 
total steps for the case n > | is therefore 4n +1. 0 
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1 void Fibonacci (int n) 
2 {4 compute the Fibonacci number F,, 


3 if (n <= 1) cout <<n << endl; /Fy =O and F, =1 
4 else { // compute F, 

5 int fn; int fum2 =0; int frm] = 15 
6 for (int i = 2; i <=; i++) 

7 { 

8 jfn=fnm1+ fam2; 

9 frm2= fam 1; 

10 fam 1= fn; 

YW } / end of for 

12 cout << fn << endl; 

13 } // end of else 


14 } 


Program 1.25: Fibonacci numbers 


Summary 

The time complexity of a program is given by the number of steps taken by 
the program to compute the function it was written for. The number of steps is 
itself a function of the instance characteristics. Although any specific instance 
may have several characteristics (¢.g., the number of inputs, the number of out- 
puts, the magnitudes of the inputs and outputs), the number of steps is computed 
as a function of some subset of these. Usually, we choose those characteristics 
that are of importance to us. For example, we might wish to know how the com- 
puting (or run) time (i.e., time complexity) increases as the number of inputs 
increase. In this case the number of steps will be computed as a function of the 
number of inputs alone. For a different program, we might be interested in deter- 
mining how the computing time increases as the magnitude of one of the inputs 
increases, In this case the number of steps will be computed as a function of the 
magnitude of this input alone. Thus, before the step count of a program can be 
determined, we need to know exactly which characteristics of the problem 
instance are to be used. These define the variables in the expression for the step 
count. In the case of Sum, we chose to measure the time complexity as a func- 
tion of the number, , of elements being added. For function Add, the choice of 
characteristics was the number, m, of rows and the number, 7, of columns in the 
matrices being added. 

Once the relevant characteristics (”, m, p, 9, r, ***) have been selected, 
we can define what a step is. A step is any computation unit that is independent 
of the characteristics (n, m, p, 9, r, °**). Thus, 10 additions can be one step; 
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100 multiplications can also be one step; but 7 additions cannot. Nor can m/2 
additions, p +g subtractions, and so on be counted as one step. 

A systematic way to assign step counts was also discussed. Once this has 
been done, the time complexity (i.e., the total step count) of a program can be 
obtained using either of the two methods discussed. 

The examples we have looked at so far were sufficiently simple that the 
time complexities were nice functions of fairly simple characteristics like the 
number of elements, and the number of rows and columns. For many programs, 
the time complexity is not dependent solely on the nurhber of inputs or outputs or 
some other easily specified characteristic. Consider the function BinarySearch 
(Program 1.10). This function searches a[0], ---,a(n—1] for x. A natural 
parameter with respect to which you might wish to determine the step count is 
the number, n, of elements to be searched. That is, we would Jike to know how 
the computing time changes as we change the number of elements n. The 
parameter n is inadequate. For the same a, the step count varies with the position 
of x in a. We can extricate ourselves from the difficulties resulting from situa- 
tions wherein the chosen parameters are not adequate to determine the step count 
uniquely by defining three kinds of step counts: best-case, worst-case, and aver- 
age, 

The best-case step count is the minimum number of steps that can be exe- 
cuted for the given parameters. The worst-case step count is the maximum 
number of steps that can be executed for the given parameters. The average step 
count is the average number of steps executed on instances with the given 
parameters. 


1.7.1.3 Asymptotic Notation (O, Q, ©) 


Our motivation to determine step counts is to be able to compare the time com- 
plexities of two programs that compute the same function and also to predict the 
growth in run time as the instance characteristics change. 

Determining the exact step count (either worst-case or average) of a pro- 
gam can prove to be an exceedingly difficult task. Expending immense effort to 
determine the step count exactly is not a very worthwhile endeavor, since the 
notion of a step is itself inexact. (Both the instructions x = y and x = 
y +z + (x/y) + (¢*y*z~x/z) count as one step.) Because of the inexactness of 
what a step stands for, the exact step count is not very useful for comparative 
purposes. An exception to this is when the difference in the step counts of two 
programs is very large, as in 3n + 3 versus 100n + 10, We might feel quite safe 
in predicting that the program with step count 3n +3 will run in less time than the 
one with step count 100n +10. But even in this case, it is not necessary to know 
that the exact step count is 100n +10. Something like, “‘it’s about 80n, or 85n, or 
75n,’’ is adequate to arrive at the same conclusion. 
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For most situations, it is adequate to be able to make a statement like ey? 
S tp(n) <= cg? or fo(nm) = cyn + com where c, and cz are nonnegative con= 
stants. This is so because if we have two programs with a complexity of cyn* + 
¢2n and c4n respectively, then we know that the one with complexity cn will be 
faster than the one with complexity c,n? + cn for sufficiently large values of n. 
For small values of n, either program could be faster (depending on cj, ¢2, and 
€3). Ife, = 1,9 = 2, and cy = 100 then cn? + can <c3n for n $98, and cin? 
+92 >c3n forn> 98. Ife, = 1,c2 =2, and cz = 1000, then cin? +cgnSean 
for n < 998. 

No matter what the values of c,, ¢2, and c3, there will be an n beyond 
which the program with complexity c3n will be faster than the one with com- 
plexity c;n? + can. This value of n will be called the break-even point, If the 
break-even point is zero, then the program with complexity ¢3n is always faster 
(or at least as fast). The exact break-even point cannot be determined analyti- 
cally. The programs have to be run on a computer in order to determine the 
break-even point. To know that there is a break-even point, it is adequate to 
know that one program has complexity c,n? + cn and the other ¢3n for some 
constants ¢;, C2, and c3. There is little advantage in determining the exact 
values of c,,¢2, and ¢3. 

With the previous discussion as motivation, we introduce some terminol- 
ogy that will enable us to make meaningful (but inexact) statements about the 
time and space complexities of a program. In the remainder of this chapter, the 
functions f and g are nonnegative functions. 


Definition [Big ‘‘oh’’}: f(n) = O(g (71) (read as “‘f of n is big oh of g of n'’) iff 
(if and only if) there exist positive constants c and mg such that f(n) < cg (”) for 
alln,n2no. 0 


Example 1.15: 3n +2 = O(n) as 3n +2 < 4n for all n> 2. 3n +3 = O(n) as 3n+ 
3 $ 4n for all n> 3. 100n + 6 = O(n) as 100n +6 < 101n forn 210. 10n? +4n+ 
2 = O(n?) as 10n? + 4n + 2.5 lin? forn 25. 1000n? + 100n - 6 = O(n2) as 
1000n? + 100n - 6 < 1001n? for n 2 100. 6*2" +n? = O(2") as 642" +n? < 
742" forn24. 3n+3 = O(n?) as 3n +3 3n? forn 22. 10n? +4n+2= On) 
as 10n? + 4n +25 10n‘ for n 22. 3n +24 O(1) as 3n + 2 is not less than or 
equal to ¢ for any constant ¢ and all n,n 2no. 10n7 +4n+24 Qn). G 


‘We write O(1) to mean a computing time that is a constant. i 
linear, O(n?) is called quadratic, O(n) is called cubic, and om ie eae 
exponential. If an algorithm takes time O(log n), it is faster, for sufficiently large 
xn, than if it had taken O(n). Similarly, O(n log n) is better than O(n?) but not as 
good as O(n). These seven computing times, O(1), O(log n), O(n), O(n log n), 
O(a*), O(a*), and O(2") are the ones we will see most often in this book. , 
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As illustrated by the previous example, the statement f(n) = O(g ()) states 
only that g(n) is an upper bound on the value of f(z) for all n,n 2 no. It does not 
say anything about how good this bound is. Notice that n = O(n?), 2 = O(n 23) n 
= O(t”), 2 = O(2"), and so on. For the statement f(a) = O(g (#)) to be informa- 
tive, g() should be as small a function of n as one can come up with for which 
F(n) = O(g(n)). So, while we shall often say 3n + 3 = O(n), we shall almost 
never say 3n + 3= O(n?), even though this latter statement is correct. 

From the definition of O, it should be clear that f(n) = O(g()) is not the 
same as O(g(n)) = f(n). In fact, it is meaningless to say that O(g (n)) = f(n). 
The use of the symbol] ‘‘="" is unfortunate because this symbol commonly 
denotes the ‘‘equals’’ relation. Some of the confusion that results from the use of 
this symbol (which is standard terminology) can be avoided by reading the sym- 
bol ‘*=”’ as ‘‘is’’ and not as ‘‘equals.’” 

Theorem 1.2 obtains a very useful result concerning the order of f(1) (i.e., 
the g(n) in f(z) = O(g (2))) when f(n) is a polynomial in n. 


Theorem 1.2: If f(n) = ayn” + --- + ayn + ao, thenf(n) = O(n”). 


Proof: f(n) < ¥ |; |n! 
i=0 
7 
sn™Z[a; | ni” 
0 
‘i 
Sn"¥|a;|,forn21 
Oo 


So, fin) = O(n"). 0 


Definition: (Omega] f(n) = Q(g(n)) (read as “*f of n is omega of g Of n’’) iff 
there exist positive constants c and ng such that f(z) 2 cg (x) for ail n,n > no. 0 


Example 1.16: 3n + 2 = Q(n) as 3n + 2 > 3n for n > 1 (actually the inequality 
holds for n 2 0, but the definition of Q requires an n9>0). 3n + 3 = Q(n) as 3n + 
3.2 3n forn2 1. 100n + 6 = Q(n) as 100n + 6 2 100n for n 21. 10n? + 4n+2= 
Qn?) as 10n? + 4n + 22 n? for n> 1. 6#2" +n? = O(2") as 642" +n? > 2" for 
n2 I. Observe also that 3n + 3 = Q(1); 10n? + 4n+2= Qn); 10n? + 4n+2= 
QC); 642" +n? = O(n"); 642" +n? = O(n); 642" +n? = O(n); 642" + n? 
= Q(n); and 6*2" +n? =Q(1). D 


As in the case of the “‘big oh’’ notation, there are several functions g(x) for 
which f(r) = O(@(2)). The function g(n) is only a lower bound on f(r). For the 
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statement f (m2) = (g (71)) to be informative, g (n) should be as large a function of 
nas possible for which the statement f() = Q(g(n)) is true. So, while we shall 
say that 3n + 3 = Q(n) and 6*2" + n? = Q(2"), we shall almost never say that 3n 
+ 3 = QI) or 6*2" +n? = Q(1), even though both of these statements are 
correct, 

Theorem 1.3 is the analogue of Theorem 1.2 for the omega notation, 


Theorem 1.3: If f(n) =ayn + -+* +a,n +a and a, > 0, then f(n) = Q(n”™). 
Proof: Left as an exercise. 0 


Definition: (Theta] f(x) = O(g (n)) (read as ‘‘f of n is theta of g of n'’) iff there 
exist positive constants c,, cz, and ng such that c)g(n) $f(n) S$ c2g (n) for all 2, 
n2ng. 0 


Example 1.17: 3n + 2 = @(n) as 3n + 2 > 3n for all n>2, and 3n + 2S 4n for alll 
n>2, 80 ¢) = 3, C2 = 4, and ng =2. 3n + 3 = O(n); 10n? + 4n + 2 = O(n”); 6*2" 
+n? = Q(2"); and 10#log n +4 = O(log n). 3n +2 # Q(1); 3n + 3 # O(n); LON? 
+4n+2# O(n); 10n? + 4n +2 # O(1); 6*2" +n? # O(n); 642" + 0? # O(n'™); 
and 6*2" +17 #Q(1). 0 


The theta notation is more precise than both the “‘big oh’’ and omega nota- 
tions. f(n) = @(g (2) iff g (2) is both an upper and lower bound on f(7). 

Notice that the coefficients in all of the g(n)’s used in the preceding three 
examples have been 1. This is in accordance with practice. We shall almost 
never find ourselves saying that 3n + 3 = O(3n), or that 10 = O(100), or that 10n2 
+ 4n +2 = Q(4n?), or that 642" + n? = Q(6*2"), or that 6*2" + n? = O(4*2"), 
even though each of these statements is true. 


Theorem 1.4: If f(n)=a,n" + «++ +a, + ag and a,, > 0, then f(a) = O(n"). 
Proof: Left as an exercise. 0 


Let us reexamine the time complexity analyses of the previous section. For 
function Sum (Program 1.17) we had determined that fsy,(n) = 22 +3. So, 
t sum (t) = OC). t Reum(tt) = 2n +2 = O(n) and tygg(m,n) = 2mn + 2n + 1 = O(n). 

Although we might all see that the O, Q, and © notations have been used 
correctly in the preceding paragraphs, we are still left with the question, Of what 
use are these notations if one has to first determine the step count exactly? The 
answer to this question is that the asymptotic complexity (i.e., the complexity in 
terms of O, Q, and ©) can be determined quite easily without determining the 
exact step count. This is usually done by first determining the asymptotic 
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complexity of each statement (or group of statements) in the program and then 
adding up these complexities. Tables 1.4 through 1.6 do just this for Sum, Rsum, 
and Add (Programs 1.17, 1.18, and 1.22). 

Note that in the table for Add (Table 1.6), lines 3 and 4 have been lumped 
together even though they have different frequencies. This lumping together of 
these two lines is possible because their frequencies are of the same order. 


line(s)__s/e___ frequency _total steps 


1 @(0) 
2 I 1 Oi) 
3 1 n+l O(n) 
4 1 n O(n) 
5 1 1 (1) 
6 0 (0) 


{ Sum(h) = O(max {g;(2)}) = Or) 
pa a as 


Table 1.4: Asymptotic complexity of Sum (Program 1.17) 


line sfe frequency total steps 
n=0  n>0 n=0 n>d 
1 0 = - 0 ©(0) 
2a) 2 1 1 1 (1) 
2(b) 1 1 0 1 (0) 
3 24 tRsum(a—1) 0 1 0) O(2 + Rsum(n—1)) 
4 0 - - 0 (0) 
tReum(A) = 2 O(2 + t Reum(tt—1)) 


Table 1.5: Asymptotic complexity of Rswm (Program 1.18) 


Although the analyses of Tables 1.4 through 1.6 are actually carried out in 
terms of step counts, it is correct to interpret tp(n) = O(g (m)), or tp(n) = O(g (2), 
or fp(1) = Q(g (n)) as a statement about the computing time of program P. This 
is so because each step takes only (1) time to execute. 

After you have had some experience using the table method, you will be in 
a position to arrive at the asymptotic complexity of a program by taking a more 
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line(s) _s/e__ frequency _ total steps 


1 0 (0) 
2 1 (m) O(m) 
3,4 1 (mn) O(mn) 
5 0 - (0) 
taag(™,n) = O(mn) 


Table 1.6; Asymptotic complexity of Add (Program 1.22) 


global approach. We elaborate on this method in the following examples. 


Example 1.18 [Permutation generator]: Consider function Permutations (Pro- 
gram 1.12), Assume that a is of size n. When k = n—1, we see that the time 
taken is O(n). When k < n—I, the else clause is entered. At this time, the second 
for loop is entered n-k times. Each iteration of this loop takes 
OltPenmutarions(k + 1, 2-1) time. So, "Permutations(ky 2-1) = 
ORK) trermutations(k + 1,2-1))) when k<n-1. Using the substitution 
method, we obtain tpenmutations(O, 2-1) = O(n(n!)),n 21. 0 


Example 1.19 [Binary search]: Let us obtain the time complexity of function 
BinarySearch (Program 1.10). The instance characteristic that we shall use is the 
number n of elements in a. Each iteration of the for loop takes O(1) time. We 
can show that the for loop is iterated at most { logo(n+1) ] times. Since an 
asymptotic analysis is being performed, we do not need such an accurate count 
of the worst-case number of iterations. Each iteration except for the last results 
in a decrease in the size of the segment of a that has to be searched by a factor of 
about 2. So, this loop is iterated {log n) times in the worst-case. As each itera- 
tion takes O(1) time, the overall worst-case complexity of BinarySearch is O(log 
n). Note that, if a was passed by value, the complexity of using BinarySearch 
would be more than this because it would take Q(n) time just to invoke the func- 
tion. O 


Example 1.20 (Magic square]: Our final example is a problem from recrea- 
tional mathematics. A magic square is an nxn matrix of the integers 1 to n? 
such that the sum of every row, column, and diagonal is the same. Figure 1.3 
gives an example magic square for the case n = 5. In this example, the common 
sum is 65. 


H. Coxeter has given the following simple rule for generating a magic 
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Figure 1.3: Example magic square 


square when n is odd: 


Start with 1 in the middle of the top row; then go up and left, assigning 
numbers in increasing order to empty squares; if you fall off the square ima- 
gine the same square as tiling the plane and continue; if a square is occu- 
pied, move down instead and continue. 


The magic square of Figure 1.3 was formed using this rule. Program 1.26 is the 
C++ program for creating an n xn magic square for the case when n is odd. This 
results from Coxeter’s rule. 

The magic square is represented using a two-dimensional array having n 
rows and n columns. For this application it is convenient to number the rows 
(and columns) from zero to n — | rather than from one to n. Thus, when the pro- 
gram “‘falls off the square,” i and/or j are set back to zero or n — 1. 

The while loop is govemed by the variable key, which is an integer vari- 
able initialized to 2 and increased by one each time through the loop. Thus, each 
statement within the while loop will be executed no more than n2 — | times, and 
the computing time for Magic is O(n"). Since there are n? positions in which the 
algorithm must place a number, we see that O(n?) is the best bound an algorithm 
for the magic square problem can have. O 
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void Magic (const int 12) 

{// Create a magic square of size n, n is odd. 
const int MaxSize = 51; // maximum square size 
int square[MaxSize [MaxSize], k, 13 


4 check correctness of n 
if ((n > MaxSize) Il (n< 1) 
throw "Error!..n out of range"; 
else if (!(%2)) throw “Error!..n is even"; 


4H nis odd. Coxeter’s rule can be used 

for (int i = 0; i<j; i++) / initialize square to 0 
Sill(square [i], square {i] + n,0); STL algorithm 

square{O]{(n-1)/2) = 1; # middle of first row 


/i and j are current position 

int key = 2; i= 0; int j =(n-1)/2; 

while (key <= 7 * n) { 

Hf move up and left 
if(-1<0)k=n-1; else 
if(G@-1<O)l= 1; else 1= 
if (square[K}[{]) i= (i4+1)%n; 1 Square occupied, move down 
else { // square [k}{1] is unoccupied 

isk j=l 


square[i][j} = keys 
key++3 
} 4 end of while 


# output the magic square 
cout << "magic square of size “ << n << endl; 
for (§ = 0; i< 3 i++) { 
copy(square [i], square [i] + n, ostream_iterator<int>(cout, " "')); 
cout << endl; 
} 
} 


Program 1.26: Magic square 
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1.7.1.4 Practical Complexities 


We have seen that the time complexity of a program is generally some function 
of the instance characteristics. This function is very useful in determining how 
the time requirements vary as the instance characteristics change. The complex- 
ity function may also be used to compare two programs P and Q that perform the 
same task. Assume that program P has complexity @() and program Q is of 
complexity @(n7). We can assert that program P is faster than program Q for 
sufficiently large n. To see the validity of this assertion, observe that the actual 
computing time of P is bounded from above by cn for some constant ¢ and for all 
n,n 2nj,, whereas that of Q is bounded from below by dn? for some constant d 
and all n,n > nz. Since cn $ dn? for n> c/d, program P is faster than program Q 
whenever 2 2 max{7), 2, c/a}. 

You should always be cautiously aware of the presence of the phrase 
“‘sufficiently large’’ in the assertion of the preceding discussion. When deciding 
which of the two programs to use, we must know whether the n we are dealing 
with is, in fact, sufficiently large. If program P actually runs in 10°n mil- 
liseconds, whereas program Q runs in n? milliseconds, and if we always have n < 
10°, then, other factors being equal, program Q is the one to use. 

To get a feel for how the various functions grow with n, you are advised to 
study Table 1.7 and Figure 1.4 very closely. It is evident from the table and the 
figure that the function 2” grows very rapidly with n. In fact, if a program needs 
2" steps for execution, then when n = 40, the number of steps needed is approxi- 
mately 1.1*10'?. On a computer performing | billion steps per second, this 
would require about 18.3 minutes. If n = 50, the same program would run for 
about 13 days on this computer. When n = 60, about 310.56 years will be 
required to execute the program and when n = 100, about 4*10!? years will be 
needed. So, we may conclude that the utility of programs with exponential com- 
plexity is limited to small » (typicatly 2 < 40). 


logn a nlogn n? a ral 

f(t) 1 0 1 1 2 
1 2 2 4 8 4 
2 4 8 16 64 16 
3 8 24 64 512 256 
4 16 64 256 4096 65,536 
5 32 160 1024 32,768 4,294,967,296 


Table 1.7: Function values 


60 Basic Concepts 


Figure 1.4: Plot of function values 


Programs that have a complexity that is a polynomial of high degree are 
also of limited utility. For example, if a program needs n'° steps, then using our 
1-billion-steps-per-second computer, we will need 10 seconds when n = 10; 
3,171 years when n = 100; and 3.17#10'3 years when n = 1000. If the program's 
complexity had been n? steps instead, then we would need | second when n = 
1000; 110.67 minutes when n = 10,000; and 11.57 days when n = 100,000. 

Table 1.8 gives the time needed by a 1-billion-steps-per-second computer 
to execute a program of complexity f(n) instructions. From a practical stand- 
point, it is evident that for reasonably large n (say n > 100), only programs of 
small complexity (such as 1, #logn, n?, n7) are feasible. Further, this is the case 
even if one could build a computer capable of executing 10!? instructions per 
second. In this case, the computing times of Table 1.8 would decrease by a 


Performance Analysis and Measurement 61 


CL S{n) 
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ps = microsecond = 10° seconds; ms = milliseconds = 10-3 seconds 
S = seconds; m = minutes; h = hours; d = days; y = years 


‘imes on a L-billion-steps-per-second computer 


factor of 1000. Now, when n = 100 it would take 3.17 years to execute n!° 
instructions and 4*10'° years to execute 2” instructions. 


1.7.2 Performance Measurement 


Performance measurement is concerned with obtaining the actual space and time 
requirements of a program. These quantities are dependent on the particular 
compiler and options used as well as on the specific computer on which the pro- 
gram is run. So, when you repeat the performance experiments reported in this 
book, the run times you will observe will be quite different. In fact, as computers 
are continuouly getting faster, your times will, most likely, be much smaller than 
those reported in this book. 

In keeping with the discussion of the preceding section, we shal! not con- 
cern ourselves with the space and time needed for compilation. We justify this 
by the assumption that each program (after it has been fully debugged) will be 
compiled once and then executed several times. Certainly, the space and time 
needed for compilation are important during program testing, when more time is 
spent on this task than in actually running the compiled code. 

We shall not explicitly consider measuring the run-time space requirements 
of a program. Rather, we shall focus on measuring the computing time of a pro- 
gram. To obtain the computing {or run) time of a program, we need a clocking 
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function. We assume the existence of a function fime (Asec) that returns in the 
variable hsec the current time in hundredths of a second. 

Suppose we wish to measure the worst-case performance of the sequential 
search function (Program 1.27). Before we can do this, we need to: (1) decide 
on the values of a for which the times are to be obtained and (2) determine, for 
each of the above values of a, the data that exhibits the worst-case behavior. 


int SequentialSearch (int *a, const int 7, const int x) 
{4 Search a [0:-1}. 


inti; 

for (§=0;i <n && ali}!= x5 i++); 
if @ = =n) return -1; 

else return i; 


} 


Program 1.27: Sequential search 


The decision on which values of n to use is to be based on the amount of 
timing we wish to perform and also on what we expect to do with the times once 
they are obtained. Assume that for Program 1.27, our intent is simply to predict 
how long it will take, in the worst-case, to search for x given the size n of a. An 
asymptotic analysis reveals that this time is O(n). So, we expect a plot of the 
times to be a straight line. Theoretically, if we know the times for any two 
values of n, the straight line is determined, and we can obtain the time for all 
other values of n from this line. In practice, we need the times for more than two 
values of n. This is so for the following reasons: 


(1) Asymptotic analysis tells us the behavior only for ‘‘sufficiently large’’ 
values of n. For smaller values of x the run time may not follow the asymp- 
totic curve. To determine the point beyond which the asymptotic curve is 
followed, we need to examine the times for several values of n. 


(2) Even in the region where the asymptotic behavior is exhibited, the times 
may not lie exactly on the predicted curve (straight line in the case of Pro- 
gram 1.27) because of the effects of low-order terms that are discarded in 
the asymptotic analysis. For instance, a program with asymptotic complex- 
ity O(n) can have an actual complexity that is c,n + ¢2logn + c3, or for 
that matter any other function of n in which the highest-order term is c\n 
for some constant, ¢),¢, > 0. 


Itis reasonable to expect that the asymptotic behavior of Program 1.27 will 
begin for some » that is smaller than 100. So, for n > 100 we shall obtain the run 
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time for just a few values. A reasonable choice is n = 200, 300, 400, , 1000. 
There is nothing magical about this choice of values. awe! can just as well use n= 
500, 1000, 1500, ---, 10,000 or n = 512, 1024, 2048, ---, 25. Ir will cost us 
more in terms of computer time to use the latter choses; id we will probably 
not get any better information about the run time of Program 1.27 using these 
choices. 

For n in the range [0, 100] we shall carry out a more refined measurement, 
since we are not quite sure where the asymptotic behavior begins. Of course, if 
our measurements show that the straight-line behavior does not begin in this 
range, we shal] have to perform a more detailed measurement in the range (100, 
200] and so on, until the onset of this behavior is detected. Times in the range 
[0, 100] will be obtained in steps of 10 beginning at n = 0. 

Tt is easy to see that Program 1.27 exhibits its worst-case behavior when x 
is chosen such that it is not one of the a[é]’s. For definiteness, we shall set a [i] 
=i,1Sisnandx=0. 

At this time, we envision using a program such as Program 1.28 to obtain 
the worst-case times. 


void TimeSearch() { 
int a[1001}, n[20]; 
for (int i al; ? <= 1000; j++) // initialize a 


Fe 10; j++) {/ values of n 
= 10 * j; n[j +10} = 100 * (f+ 1); 


cout <<" n time" << endl; 
for = 0; j < 20; j++) { / obtain computing times 
tong start, stop; 
time (start) ; // start timer 
int k = SequentialSearch(a, n [j], 0); // unsuccessful search 
time (stop) ; / stop timer 
long runTime = stop — start ; 
cout<<" "<<n[j)<<" “<< runTime << endl; 


cout << "Times are in hundredths of a second.” << endl ; 


} 
Program 1,28: Program to time Program 1.27 


The output obtained from this program is summarized in Figure 1.5. All 
the times are zero, indicating that the precision of our clock is inadequate. 
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n time n time 

0 0 100 it) 
10 i) 200 0 
20 0 300 0 
30 0 400 0 
40 0 500 0 
50 0 600 0 
60 0 700 0 
70 0 800 0 
80 0 900 0 
90 0 1000 0 


Times in hundredths of a second 
Figure 1.5: Output from Program 1.28 


To time a short event, it is necessary to repeat it several times and divide 
the total time for the event by the number of repetitions. 

Since our clock has an accuracy of about one-hundredth of a second, we 
should not attempt to time any single event that takes less than about 1 second. 
With an event time of at least 1 second, we can expect our observed times to be 
accurate to | percent. 

The body of Program 1.28 needs to be changed to that of Program 1.29. In 
this program, r[j] is the number of times the search is to be repeated when the 
number of elements in the array is n[j]. Notice that rearranging the timing state- 
ments as in Programs Program 1.30 or Program 1.31 does not produce the desired 
results. For instance, from the data of Figure 1.5, we expect that with the struc- 
ture of Program 1.30, the value output will still be 0 because in each iteration of 
the for loop, 0 gets added to soral. With the structure of Program 1.31, we expect 
the program never to exit the for loop. Yet another altemative is to move the 
first call to time out of the for loop of Program 1.31 and change the assignment to 
total within the for loop to 


long total = stop — start; 


This approach can be expected to yield satisfactory times. This approach cannot 
be used when the timing function available gives us only the time since the last 
invocation of time. Another difficulty is that the measured time includes the time 
needed to read the clock. For small n, this time may be larger than the time to 
run SequentialSearch. This difficulty can be overcome by determining the time 
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taken by the timing function and subtracting this time later. In further discus- 
sion, we shall use the explicit repetition factor technique. 


void TimeSearch() { 
int a[1001], n[20}; 
const long 1{20} = {300000, 300000, 200000, 200000, 100000, 100000, 


100000, 80000, 80000, 50000, 50000, 25000, 15000, 15000, 10000, 7500, 7000, 


6000, 5000, 5000}; 


for (int j = 1; j <= 1000; j++) / initialize a 
allah 


for (j =0; j < 10; j++) {/ values ofn 
n[j] = 10 * j 3 nj +10] = 100 * G+); 
} 


cout << " n totalTime runTime" << endl; 


for(j = 0; j < 20; j++) { / obtain computing times 
long start, stop ; 
time (start); // start timer 
for (long b = 1; 6 <=r[j); b++) 
int k = SequentialSearch(a, n[j), 0); // unsuccessful search 
time (stop) ; ! stop timer 
long toralTime = stop — start ; 
float runTime = (float) (totalTime)i(float){r [j }) 5 
cout <<" "<<a[j]<<" "<< totalTime <<" " << runTime << endl; 


cout << "Times are in hundredths of a second.” << endl; 


} 


Program 1.29: Timing program 


The output from the timing program, Program 1.29, is given in Figure 1.6. 
The times for n in the range [0, 100] are plotted in Figure 1.7. The remaining 
values have not been plotted because this would lead to severe compression of 
the range {0, 100]. The linear dependence of the worst-case time on n is 
apparent from this graph. 

The graph of Figure 1.7 can be used to predict the run time for other values 
of n. For example, we expect that when n = 24, the worst-case search time will 
be 0.0031 hundredths of a second. We can go one step further and get the 
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long total = 0; 
for (long b = 1; b<=r[j]; b++) 
{ 


time (start); 

k = SequentialSearch (a, n|j], 0); 
time(stop); 

total += stop — start; 


float runTime = (float) (total) / (leat) (r [7 1); 


Program 1.30: Improper timing construct 


for(long total = 0, i = 0; total < DesiredTime; i++) 
{ 


time (start); 

k = SequentialSearch (a,n [j],0); 
time (stop); 

total += stop — start; 


float runTime = (float) (total) / (Moat) (i); 


Program 1.31: Another improper timing construct 


equation of the straight line. The equation of this line is ¢ = ¢ + mn, where m is 
the slope and c the value for n = 0. From the graph, we see that ¢ = 0.0008. 
Using the point n = 60 and ¢ = 0.0066, we obtain m = (t-cY/n = 0.0058/60 = 
0.000096. So, the line of Figure 1.7 has the equation ¢ = 0.0008 + 0.000096n, 
where 1 is the time in hundredths of a second. From this, we expect that when n 
= 1000, the worst-case search time will be 0.0975 hsec, and when n = 500, it will 
be 0.0491 hsec, Compared with the actual observed times of Figure 1.6, we see 
that these figures are very accurate! 

An alternate approach to obtain a good straight line for the data of Figure 
1.6 is to obtain the straight line that is the least-squares approximation to the 
data. The result is t = 0.00085454 + 0.00009564n. When n = 1000 and 500, this 
equation yields t= 0.0966 and 0.0487. 

Now, we are probably ready to pat ourselves on the back for a job well 
done. However, this action is somewhat premature, since our experiment is 
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n total runTime n total runTime 

0) al 0.0008 100 527 0.0105 
10 533 0.0018 200 505 0.0202 
20 582 0.0029 300 451 0.0301 
30 736 0.0037 400 593 0.0395 
40 467 0.0047 500 494 0.0494 
50 565 0.0056 600 439 0.0585 
60 659 0.0066 700 484 0.0691 
70 604 0.0075 800 467 0.0778 
80 681 0.0085 900 434 0.0868 
90 472 0.0094 1000 484 0.0968 


Times in hundredths of a second 


Figure 1.6: Worst-case run times for Program 3.27 


Figure 1.7: Plot of the data in Figure 1.6 
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flawed. First, the measured time includes the time taken by the repetition for 
loop. So, the times of Figure 1.6 are excessive. This can be corrected by deter- 
mining the time for each iteration of this statement. A quick test run indicates 
that 300,000 executions take only 50 hundredths of a second. So, subtracting the 
time for the for (b = 1; b <= r[j]; b++) statement reduces the reported times by 
only 0.000016. We can ignore this difference, since the use of a higher repetition 


factor could well result in measured times that are lower by about 0.000016 hsec 
per search. 


Summary 


To obtain the run time of a program, we need to plan the experiment. The 
following issues need to be addressed during the planning stage: 


(1) What is the accuracy of the clock? How accurate do our results have to 


be? Once the desired accuracy is known, we can determine the length of 
the shortest event that should be timed. 


(2) For each instance size, a repetition factor needs to be determined. This is 


to be chosen such that the event time is at least the minimum time that can 
be clocked with the desired accuracy. 


Are we measuring worst-case or average performance? Suitable test data 
need to be generated. 


(4) What is the purpose of the experiment? Are the times being obtained for 
comparative purposes, or are they to be used to predict actual run times? If 
the latter is the case, then contributions to the run time from such sources 
as the repetition loop and data generation need to be subtracted (in case 
they are included in the measured time). If the former is the case, then 
these times need not be subtracted (provided they are the same for all pro- 
grams being compared). 
In case the times are to be used to predict actual run times, then we need to 
fit a curve through the points. For this, the asymptotic complexity should 
be known. If the asymptotic complexity is linear, then a least-squares 
straight line can be fit; if it is quadratic, then a parabola is to be used (i.e., t 
=a +a\n+ayn°). If the complexity is O(mlogn), then a least-squares 
curve of the form t= ag + an + aznlogzn can be fit. When obtaining the 
least-squares approximation, one should discard data corresponding to 


“small"’ values of n, since the pro; does not exhibit i i 
behavior for these a. program exhibit its asymptotic 


Q) 


(5) 
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17.3. Generating Test Data 


Generating a data set that results in the worst-case performance of a program is 
not always easy. In some cases, it is necessary to use a computer program to 
generate the worst-case data. In other cases, even this is very difficult. In these 
cases, another approach to estimating worst-case performance is taken. For each 
set of values of the instance characteristics of interest, we generate a suitably 
large number of random test data. The run times for these test data are obtained. 
The maximum of these times is used as an estimate of the worst-case time for 
this set of values of the instance characteristics. 

To measure average-case times, it is usually not possible to average over 
al} possible instances of a given characteristic. Although it is possible to do this 
for sequential and binary search, it is not possible for a sort program. If we 
assume that all keys are distinct, then for any given n, n! different permutations 
need to be used to obtain the average time. Obtaining average-case data is usu- 
ally much harder than obtaining worst-case data. So, we often adopt the strategy 
outlined above and simply obtain an estimate of the average time. 

Whether we are estimating worst-case or average time using random data, 
the number of instances that we can try is generally much smaller than the total 
number of such instances. Hence, it is desirable to analyze the algorithm being 
tested to determine classes of data that should be generated for the experiment. 
This is a very algorithm-specific task, and we shall not discuss it here. 


EXERCISES 


1. Compare the two functions n? and 2/4 for various values of n. Determine 
when the second becomes larger than the first. 

2. Prove by induction: 
@) > i=n(n+i2,n21 


Isis 

(&) YS i? =nln + 1)Qn 4+ 16,221 
isisn 

©) xh=@"*! 1-1, x #1020 
Osisn 


3, Determine the frequency counts for all statements in the following two pro- 
gram segments: 
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ny, i++) lish; 
j<=i; j++) 2 while (i Sn) 

3 ik <= fi ket) 3{ 

4 4 x++; 
5 food 
6} 

@) {b) 
4. (a) Introduce statements to increment count at all appropriate points in 
Program 1.32. 


void D(int *x, int n) 
{ 
inti=1; 
do { 
xi} +=2; 
i+=2; 
while (i <= n); 
f=]; 


while (i <= (n /2)) 
{ 


xfij+= xfi+i)s 
i++5 


} 
Program 1.32: Example program 


(b) Simplify the resulting program by eliminating statements. The 
simplified program should compute the same value for count as com- 
puted by the program of (a). 

{c) What is the exact value of count when the program terminates? You 
may assume that the initial value of count is 0. 

(d) Obtain the step count for Program 1.32 using the frequency method. 
Clearly show the step count table. 

5. Do Exercise 4 for function Transpose (Program 1.33). 


Do Exercise 4 for Program 1.34. This program multiplies two nxn 
matrices a and b. 
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void Transpose(int **a3 int n) 


for (int i= 03 i <n—-1; i++) 
for (int j = i+1; j <n; j++) 
' swap (a(i}Ui], aU MED; 


Program 1.33: Matrix transpose 


void Multiply( int **a, int **b, int **c, int n) 


for (int i= 03 i <n; i++) 
for (int j = 0; j <n; j++) 
{ 


cl] =0; 
for (int k= 0;k <n; k++) 
efi)U) += 2 [é)[k) * (AIG); 


} 
Program 1.34: Square matrix multiplication 


7. (a) Do Exercise 4 for Program 1.35. This program multiplies two 
matrices a and b where a is an m X n matrix and 6 is an n x p matrix. 


(b) Under what conditions will it be profitable to interchange the two 
outermost for loops? 


8. Show that the following equalities are correct: 
(a) 5n? — 62 = O(n?) 
(0) n! = O(n") 4) 
(c) 2n?2" + nlogn = O(n?2") 


(d) EP = Q(n3) 
f=0 

©) EP = en") 
i=0 


(fn™ + 6#2" = (277) 
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void Multiply(int **a, int **b, int **c, int m, int n, int p) 


for (int i = 0; i<m 3 i++) 
for (int j = 0; j <p 3 j++) 
{ 


c(fU) =0; 
for (int k = 03k <n 3 k++) 
cl) += ali ]k) * OTK Us 


} 


Program 1.35: Matrix multiplication 


(g)n? + 10°n? = O(n3) 
(h) 6n?Alogn + 1) = O(n) 
(i) n!! + plogn = O(n! ™!) 
Gj) nk *® + n*logn = O(n* *®) for all k and €,k 20, ande>0 
(k) 10n? + 152 + 100n22" = O(100n22") 
(1) 33n? + 4n? = Qn?) 
(m) 33n? + 4n? = O(n3) 
9. Show that the following equalities are incorrect: 
(a) 10n?+9 = O(n) 
(b) n?logn = O(n?) 
(c) n?Aogn = O(n?) 
(a) 032" + 6n?3" = O(n 2") 


10. Obtain the average run time of function BinarySearch (Program 1.10). Do 
this for suitable values of x in the range [0, 100]. Your report must include 
a plan for the experiment as well as the measured times. These times are to 


be provided both in a table and as a graph. 


11. Analyze the computing time of function SelectionSort (Program 1.8). 


12. 


Obtain worst-case run times for function SelectionSort (Program 1.8). Do 


this for suitable values of n in the range [0, 3000]. Your report must 
include a plan for the experiment as well as the measured times. These 


limes are to be provided both in a table and as a graph. 
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13. Consider function Add (Program 1.22). 
(a) Obtain run times for 2 = 100, 200, -- +, 3000. 
(b) Plot the times obtained in part (a). 
14, Do the previous exercise for matrix multiplication (Program 1.35). 


15. A complex-valued matrix X is represented by a pair of matrices (A,B) 
where A and B contain real values. Write a program that computes the pro- 
duct of two complex-valued matrices (A,B) and (C,D), where 
(A,B) # (CD) = (A + iB) *(C + iD) = (AC - BD) + {AD + BC). Deter- 
mine the number of additions and multiplications if the matrices are all 
nxn. 


16. Function Magic (Program 1.26) uses a 51 x 5] array square independent of 
the value of n. When n < SI, excess space is used and when n > 51, the 
function throws an exception. We can eliminate these shortcomings by 
using a dynamically allocated n x n 2-dimensional array. Modify function 
Magic to do this. Test your code. 
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CHAPTER 2 


Arrays 


2.1 ABSTRACT DATA TYPES AND THE C++ CLASS 


2.1.1 An Introduction to the C++ Class 


C++ provides an explicit mechanism, the class, to support the distinction 
‘between specification and implementation and to hide the implementation of an 
ADT from its users. However, it is the programmer’s responsibility to use the 
class mechanism judiciously so that it does, in fact, represent an ADT. The C++ 
class consists of four components (Program 2.1): 


{1} A class name: (e.g., Rectangle). 


{2) Data members: the data that makes up the class (e.g., xLow, yLow, height, 
and width). 


(3) Member functions: the set of operations that may be applied to the objects 
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of aclass (¢.g., GetHeight(), GetWidth() ). 


(4) Levels of program access: these control the level of access to data members 
and member functions from program code that is outside the class. There are 
three levels of access to class members: public, protected, and private. 

Any public data member (member function) can be accessed (invoked) 
from anywhere in the program. A private data member (member function) can 
only be accessed (invoked) from within its class or by a function or a class that is 
declared to be a friend. A protected data member (member function) can only 
be accessed (invoked) from within its class or from its subclasses or by a friend. 
We will discuss subclasses when we study inheritance in Chapter 3. 


#ifndef RECTANGLE_H 
#define RECTANGLE_H 
In the header file Rectangle.h 
class Rectangle { 
public: #f the following members are public 
# the next four members are member functions 
Rectangle(); /f constructor 
“Rectangle(); H destructor 
int GetHeightQ; —_‘// returns the height of the rectangle 
int GerWidth0; 4 returns the width of the rectangle 
private:  // the following members are private 
#/ the following members are data members 
int xLow, yLow, height, width; 
H (xLow, yLow) are the coordinates of the bottom left comer of the rectangle 
oF 
#endif 


Program 2.1: Definition of the C++ class Rectangle 


2.1.2 Data Abstraction and Encapsulation in C++ 


Data encapsulation is enforced in C++ by declaring all data members of a class 
to be private (or protected). External access to data members, if required, can 
be achieved by defining member functions that get and set data members. In Pro- 
gram 2.1, GetHeight() and GetWidth() are used to access the private data 
members height and width. Member functions that will be invoked externally 
are declared public; all others are declared private or protected. 
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Next, we discuss how the specification of the operations of a class is 
separated from their implementation in C++. The specification, which must be 
contained inside the public portion of the class definition of the ADT, consists of 
the names of every public member function, the type of its arguments, and the 
type of its result (This information about a function is known as its function pro- 
totype). There should also be a description of what the function does, which 
does not refer to the internal representation or implementation details. This 
requirement is quite important because it implies that an abstract data type is 
implementation-independent. This description may be achieved in C++ by using 
comments to describe what each member function does (like the DVD instruc- 
tion manual mentioned in Chapter 1). Finally, the specification of an operation is 
physically separated from its implementation by placing it in an appropriately 
named header file (e.g., the contents of Program 2.1 are placed in Rectangle.h). 
The implementations of the functions are typically placed in a source file of the 
same name (e.g., the contents of Program 2.2 are placed in Rectangle.cpp). Note 
that C++ syntax does allow you to include the implementation of a member func- 


tion inside its class definition. In this case, the function is treated as an inline 
function, 


In the source file Rectangle.cpp 
#include “Rectangle.h" 


Hf The prefix "Rectangle::" identifies GetHeightQ) and GetWidth() as member 
i functions belonging to class Rectangle. It is required because the member 
i functions are implemented outside their class definition 


int Rectangle::GetHeight() { return height;} 
int Rectangle::GetWidthQ { return width;} 


Program 2.2: Implementation of operations on Rectangle 


2.1.3 Declaring Class Objects and Invoking Member Functions 


Program 2.3 shows a fragment of code that illustrates how class objects are 
declared and how member functions that operate on these class objects are 
invoked. Class objects are declared and created in the same way that variables 
are declared or created. Members of an object are accessed or invoked by using 
the component selection operators, a dot (.) for direct component selection and 
an arrow (->), which we shall write as —, for indirect component selection 


through a pointer. 


4 Ina source file main.cpp 
#include <iostream> 
#include “Rectangle.h” 


main(){ 
Rectangle r, 5; MW rand s are objects of class Rectangle 
Rectangle #1 = &s; if Lis a pointer to class object s 


use «to access members of class objects. 

Huse -> to access members of class objects through pointers. 

if (r.GetHeight () * r.GetWidth () > t >GetHeight () * 1>GerWidth ()) 
cout << "1"; 

else cout <<"s"; 

cout << "has the greater area” << endl; 


} 


Program 2.3: A C++ code fragment demonstrating how Rectangle objects are 
declared and member functions invoked 


2.1.4 Special Class Operations 


Constructors and Destructors: The constructor and destructor are special 
member functions of a class. A constructor is a member function which initial- 
izes data members of an object. If a constructor is provided for a class, it is 
automatically executed when an object of that class is created. If a constructor is 
not defined for a class, memory is allocated for the data members of a class 
object, when it is created, but the data members are not initialized. The advan- 
tage of defining constructors for a class is that all class objects are well-defined 
as soon as they are created. This eliminates errors that result from accessing an 
undefined object. A destructor is a member function which deletes data 
members immediately before the object disappears. A constructor and destructor 
for class Rectangle are declared in Program 2.1. 

A constructor must be declared as a public member function of its class, 
The name of a constructor must be identical to the name of the class to which it 
belongs; and a constructor must not specify a retum type or return a value. Pro- 
gram 2.4 shows a constructor definition for class Rectangle. 
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Rectangle::Rectangle( int x, int y, int h, int w) 
4 
xLow = x; yLow = y; 
height = h; width = w; 
} 


Program 2.4: Definition of a constructor for Rectangle 


Constructors may be used to initialize Rectangle objects as follows: 


Rectangle r (1,3, 6, 6); 
Rectangle *s = new Rectangle (0, 0, 3. 4)3 


These create r, a square of side 6 whose bottom left comer is at (1, 3); and s, 


a 
pointer to a Rectangle object of height 3 and width 4 whose bottom left corner is 
at the origin. Note that the declaration 


Rectangle t; 


will result in a compile time error. The reason is that the compiler requires a 
default constructor (a constructor with no arguments) to initialize 1. Had we not 
defined the constructor of Program 2.4, the compiler would have generated its 
own default constructor. However, since we have defined a constructor, it is our 
responsibility to provide a default constructor if we wish to use the above 
declaration. Program 2.5 shows a definition of the Rectangle constructor that 
also serves as a default constructor. This is achieved by providing a default value 
for each argument in the argument list. In this case, the default for each argument 
is the integer 0. If this constructor is used, the above declaration for object r will 


result in the creation of a Rectangle object, all of whose data members are initial- 
ized to 0. 


Rectangle::Rectangle( int x = 0, int y = 0, int h = 0, int w = 0) 
: xLow (x), yLow (y), height (h), width (w) 
{} 


Program 2.5: Sophisticated definition of a constructor for Rectangle 
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The constructor of Program 2.5 also differs from that of Program 2.4 in another 
Tespect: its body is empty; the data members are initialized by using a member 
initialization list (consisting of a colon followed by a list of data members and 
the arguments to which they are to be initialized in parentheses). Program 2.4 
first initializes the data members and then assigns arguments to them in two 
separate steps, while Program 2.5 directly initializes the data members to the 
corresponding arguments in a single step. Thus, the latter approach results in a 
more efficient constructor. 

Destructors are automatically invoked when a class object goes out of 
scope or when a class object is deleted. Like a constructor, a destructor must be 
declared as a public member of its class; Its name must be identical to the name 
of its class prefixed with a tilde, ~; a destructor must not specify a retum type or 
return a value, and a destructor may not take arguments. If a destructor is not 
defined for a class, the deletion of an object of that class results in the freeing of 
memory associated with data members of the class. If a data member is a pointer 
to some other object, the space allocated to the pointer is returned, but the object 
that it was pointing to is not deleted. If we also wish to delete this object, we 
must define a destructor that explicitly does so. 


Operator Overloading: Consider the operator == which is used to check for 
equality between two data items. The operator == may be used to check for 
equality between two float data items; it can also be used to check for equality 
between two int items. The hardware algorithms implementing operator = 
depend on the type of the operands being compared; that is, the algorithm for 
comparing two floats is different from the one used to compare two ints. This is 
an example of operator overloading. However, if we were to try to use operator 
== to check for equality between two Rectangle objects, the compiler would 
complain that operator = is not defined for Rectangle objects. C++ allows the 
programmer to overload operators for user-defined data types. This is done by 
providing a definition that implements the operator for the particular data type. 
This definition takes the form of a class member function or an ordinary function, 
depending on the operator. The function prototype used must adhere to the 
specifications for the particular operator. For details about the specifications for 
various operators, see one of the introductory texts listed at the end of this 
chapter. 

Program 2.6 overloads operator == for class Rectangle. Our program uses 
the this pointer which we describe briefly: The C++ keyword this, when used 
inside a member function of a class, represents a pointer to the particular class 
object that invoked it. The class object, itself, is therefore represented by *this. 


‘We can now use the operator == to determine whether two rectangles are identi- 
cal. Our program first evaluates the expression "this == &s". This expression 
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bool Rectangle::operator== (const Rectangle& s) 


if (this == &s) return true; 
if ((xLow == s.xLow) && (yLow == s.yLow) 

&& (height == s.height) && (width == s. width) ) return true; 
else return fal 


} 
Program 2.6: Overloading operator== for class Rectangle 


checks to see if the two rectangles being compared are the same object. This 
would happen in the following two cases: 


if(@==r) ++: 


In both cases, the expression “this == &s" evaluates to true because both 
operands represent the same object, and there is no need to compare the indivi- 
dual data members of the two rectangles. This is especially efficient if the class 
contains a large number of data members. If the expression evaluates to false, 
then the two rectangles are not the same object, and the individual data members 
must be compared. Program 2.7 overloads operator << so that Rectangle objects 
can be output by using cout. 


ostream& operator<< (ustream& os, Rectangle & r) 
os << “Position is: "<< r.xLow <<"""5 
os << r.yLow << endl; 
os << “Height is: " << r.height << endl; 
os << "Width is: “ << r. width << endl; 
return os; 


} 
Program 2.7: Overloading operator<< for class Rectangle 


Abstract Data Types and the C++ Class 81 


Notice that operator<< accesses private data members of class Rectangle. 
Therefore, it must be made a friend of Rectangle. In general, we want to minim- 
ize the number of friend declarations in a class because a friend represents an 
exception to the data encapsulation principte. However, there are instances, such 
as this, when they are necessary. The statement: 


cout << r5 


where ris defined as a square of side 6 with bottom left comer at (1, 3), will print 
the following: 


Position is: 13 
Height is: 6 
Width is: 6 


2.1.5 Miscellaneous Topics 


In C++, a struct is identical to a class, except that the default level of access is 
public; that is, if the struct definition of a data type does not specify whether a 
given member (data or function) has public, private, or protected access, then 
the member has public access. In a class, the default is private access. Thus, the 
C++ struct is a generalization of the C struct. 


A union is a structure that reserves storage for the largest of its data 
members so that only one of its data members can be stored, at any time. This is 
useful in applications where it is known that only one of many possible data 
items, each of a different type, needs to be stored in a structure; but there is no 
way to know what that data type is unti) runtime. The struct or class structures 
reserve memory for all their data members. Thus, using a union results in a more 
memory-efficient program, in these cases. We wil! use union in Chapter 4 on 
linked lists; we will also study a technique for improving on union by using 
inheritance. 


A static class data member may be thought of as a global variable for its 
class. From the perspective of a class member function, a static data member is 
like any other data member. One difference is that each class object does not 
have its own exclusive copy. There is only one copy of a static data member and 
all class objects must share it. A second difference is that the declaration of a 
static data member in its class does not constitute a definition. Consequently, a 
definition of the data member is required somewhere else in the program. We 
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will see an example of a static class member later in this chapter, when we 
implement the Polynomial data type. 


2.1.6 ADTs and C++ classes 


Example 2.1 [Abstract data type NaturalNumber}: ADT 2.1 contains the class 
definition of NaturalNumber. You will notice that the class definition of 
NaturalNumber in ADT 2.1 is very similar to the ADT definition of ADT 1.1. Ag 
a result, we will henceforth use the C++ class to define an ADT instead of the 
notation of ADT 1.1. Note that there is one significant aspect in which the format 
of ADT 1.1 differs from the C++ class: some operators in C++ such as opera- 
tor<<, when overloaded for user-defined ADTs, do not exist as member func- 
tions of the corresponding class. Rather, these operators exist as ordinary C++ 
functions, Thus, these operations are declared outside the C++ class definition of 
the ADT even though they are actually part of the ADT. 


class NaturalNumber { 


J An ordered subrange of the integers starting at zero and ending at 


4 the maximum integer (MAXINT) on the computer. 
public: 


NaturalNumber Zero( ); 
H Returns 0. 


bool /sZero(); 
Kf *this is 0, return true; otherwise, return false. 


NaturalNumber Add(NaturalNumber y); 
# Retum the smaller of *this + y and MAXINT. 


bool Equal(NaturalNumber y); 
# Retum true if *this == y; otherwise return false. 


NaturalNumber Successor( ); 
# Mf *this is MAXINT return MAXINT; otherwise return *this + 1. 


NaturalNumber Subtract(NaturalNumber ys 
JAF *this < y, return 0; otherwise return *this — y. 


' 


ADT 2.1: Abstract data type NaturalNumber 
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EXERCISES 


1 


2. 


Overload operator< for class Rectangle such that r < s if and onily if the 
area of r is less than that of s. 


Write and test C++ code for the class MyRectangle, which is an enhanced 
version of the class Rectangle (Program 2.1). In addition to the data 
members listed in Program 2.1, MyRectangle has the data member color. 
You must include functions to change as well as to return the value of each 
data member and functions to return the area and perimenter of a rectangle. 
The operators << and >> should be overloaded to work with rectangles. 


Write and test code for the C++ class Currency, which represents currency 
objects. Each currency object has two data members $ and cents, where 
cents is an integer between 0 and 99. You must include functions to set and 
return the data members, add and subtract currency objects, to multiply a 
currency object by numbers such as 2, 2.5 and so on. The operators << and 
>> should be overloaded to work with currency objects. 


Implement a class Complex, which represents the Complex Number data 
type. Implement the following operations: 


(a) A constructor (including a default constructor which creates the 
complex number 0 + Oi). 


(b) Overload operator+ to add two complex numbers. 
(c) Overload operator* to multiply two complex numbers. 
(d) Overload << and >> to print and read complex numbers. To do this, 


you will need to decide what you want your input and output format 
to look like. 


Write a program according to the following specifications: use the con- 
structor to define two complex numbers: 3 + 2i and 0 + Oi. Input two com- 
plex numbers 5 + 3/ and 0 + 0i using cin. Obtain the sum and product of all 
four complex numbers using operators + and *, respectively. Output the 
results using cout. 


Implement a class Quadratic that represents 2-degree polynomials i.e., 

polynomials of type ax?+bx+c. Your class will require three data members 

corresponding to a, b, and c. Implement the following operations: 

(a) A constructor (including a default constructor which creates the 0 
polynomial). 

(b) Overload operator+ to add two polynomials of degree 2. 

(c) Overload << and >> to print and read polynomials. To do this, you 


will need to decide what you want your input and output format to 
look like. 
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(a) A function Eval that computes the value of a polynomial for a given 
value of x. 


(©) A function that computes the two solutions of the equation 


ax?+bx+c = 0, [Hint: use class Complex from the previous exer. 
cise.] 


2.2 THE ARRAY AS AN ABSTRACT DATA TYPE 


We begin our discussion by considering an array as an ADT. This is not the 
usual perspective, since many programmers view an array only as a consecutive 
set of memory locations. This is unfortunate because it clearly shows an 
emphasis on implementation issues. Although an array is usually implemented 
as a consecutive set of memory locations, this is not always the case. Intuitively, 
an array is a set of pairs, <index, value>, such that each index that is defined has 
a value associated with it. In mathematical terms, we call this a correspondence 
or a mapping. However, when considering an ADT, we are more concerned with 
the operations that can be performed on an array. Aside from creating a new 
array, most languages provide only two standard operations for arrays, one that 
retrieves a value and one that stores a value. ADT 2.2 shows a class definition of 
the array ADT. 

The constructor GeneralArray(int j, RangeList list, float initValue = 
defaultValue) produces a new array of the appropriate size and type. All of the 
items are initially set to the floating point variable initValue. Retrieve accepts an 
index and retums the value associated with the index if the index is valid or an 
estor if the index is invalid. Store accepts an index, and a value of type float, and 
replaces the <index, oldvalue> pair with the <index, newvalue> pair. 

GeneralArray is more general than the C++ array as it is more flexible 
about the composition of the index set. The C++ array requires the index set to 
be a set of consecutive integers starting at 0. Also, C++ does not check an array 
index to ensure that it belongs to the range for which the array is defined. 
Because of these disadvantages, it is often beneficial to define a more general 
and robust array class. Some of the member functions required to implement 
such a class are examined in the exercises. A general array class would typically 
be internally implemented by using the C++ array. To do this, it is necessary to 
be able to map a position in the general array onto a position in the C++ array. 
This is discussed in Section 2.5. 

To simplify the discussion, we will hereafter use the C++ array. Therefore, 


before continuing, let us briefly review the C++ array. A C++ array floatArray of 
floats with index set ranging from 0 to n - | is defined by: 
float floatArray{n| ; 


The ith element of floatArray may be accessed in two ways: floatArrayli] 
and *(floatArray + i). In the latter mechanism, the variable floatArray is actually 
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class GeneralArray { 
4 A set of pairs <index, value> where for each value of index in IndexSet 
H there is a value of type float . indexSet is a finite ordered set of one or more 
# dimensions, for example, (0, --- , 2-1} for one dimension, ((0, 0), 
(0, 1), (CO, 2), (1, 0), (1, 1), C1, 2), (2, 0), (2, 1), (2, 2)} for two dimensions, etc. 
public: 
GeneralArray(int j, RangeList list, float initValue = defaultValue); 
1 This constructor creates a j dimensional array of floats; the 
df range of the kth dimension is given by the kth element of /isr. For each 
H index i in the index set, insert <i, initValue> into the array. 


float Retrieve(index i); 
I€ iis in the index set of the array, return the float associated with i 
//in the array; otherwise throw an exception. 


void Store(index i, float x); 
If 7 is in the index set of the array, replace the old value associated with i 
i by x; otherwise throw an exception. 

hal 


ADT 2.2: Abstract data type GeneralArray 


a pointer to the zeroth element of the array. The expression floatArray + i is a 
pointer to the ith element of array floatArray; it follows that *(floatArray + i) is 
the ith element of array floatArray. 


EXERCISES 


1. Implement a class CppArray which is identical to a one-dimensional C++ 
array (i.e., the index set is a set of consecutive integers starting at 0) except 
for the following: 


(i) It performs range checking. 


{ii) It allows one array to be assigned to another array through the use of 
the assignment operator (e.g., cp 1 = cp2). 


(iii) It supports a function that returns the size of the array. 


(iv) It allows the reading or printing of arrays through the use of cin and 
cout. 


To do this, you will have to define the following member functions on 
Cpparray: 
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(a) 


(b) 


() 


@) 
) 


) 
(g) 


A constructor CppArray (int size = defaultSize, float initvalue = 
defaultValue). This creates an array of size size all of whose ele. 
ments are initialized to initvalue. 


A copy constructor CppArray (const CppArray& cp2). This Creates 
an array identical to cp2. Copy constructors are used to initialize a 


class object with another class object as in the following: 
CppArray a=b3 


An assignment operator CppArray& operator= (const CppArray& 
cp2). This replaces the original array with cp2. 

A destructor “CppArray(). 

The subscript operator float& operator[ ] (int i). This should be 
implemented so that it performs range-checking. 

A member function int GerSize(). This returns the size of the array. 


Implement functions to read and print the elements of a CppArray by 


overloading operators << and >>. These functions will have to be 
made friends of CppArray. 


2.3 THE POLYNOMIAL ABSTRACT DATA TYPE 


Arrays are not only data structures in their own right; we can also use them to 
implement other abstract data types. For instance, let us consider one of the sim- 


plest and most common data structures: the ordered, or linear, list. We can find 
many examples of this data structure, including: 


Days of the week: (Sunday, Monday, Tuesday, Wednesday, Thursday, Fri- 


day, Saturday) 


Values in a deck of cards: (Ace, 2, 3, 4, 5, 6,7, 8, 9, 10, Jack, Queen, King) 
Floors of a building: (basement, lobby, mezzanine, first, second) 

Years the United States fought in World War II: (1941, 1942, 1943, 1944, 
1945) 


‘Years Switzerland fought in World War II: () 


Notice that “Years Switzerland fought in World War Il’? is different 
because it contains no items. It is an example of an empty list, which we denote 


aS (). The other lists all contain items that are written in the form (a 0415 
Gat). 


th 


We can perform many operations on lists, including: 


Find the length, 1, of the list. 
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(2) Read the list from left to right (or right to left). 

(3) Retrieve the ith element, OSi <7. 

(4) Store a new value into the ith position, 0S i <n. 

(5) Insert a new element at the position i, OS i <x, causing elements num- 


bered i, i + 1, ---,a2—1 to become numbered i + 1,i+2,+--,n. 
(6) Delete the element at position i, OSi <n, causing elements numbered 
it, -++,a~ 1 to become numbered i, i + 1, +-+,2-2. 


It is not always necessary to be able to perform all of these operations; often a 
subset will suffice. In the study of data structures we are interested in ways of 
representing ordered lists so that these operations can be carried out efficiently. 

Rather than state the formal specification of the ADT ordered list, we want 
to explore briefly its implementation. Perhaps the most common way to 
represent an ordered list is by an array where we associate the list element a; 
with the array index i. We will refer to this as sequential mapping because, using, 
the conventional array representation, we are storing a, and a; , ; into consecu- 
tive locations i and i + | of the array. This gives us the ability to retrieve or 
modify the values of random elements in the list in a constant amount of time, 
essentially because a computer has random access to any word in its memory. 
We can access the list element values in either direction by changing the sub- 
script values in a controlled way. Only operations (5) and (6) require real effort. 
Insertion and deletion using sequential allocation force us to move some of the 
remaining elements so that the sequential mapping is preserved in its proper 
form. It is precisely this overhead that leads us to consider nonsequential map- 
pings of ordered lists in Chapter 4. 

Let us jump right into a problem requiring ordered lists, which we shall 
solve by using one-dimensional arrays. The problem calls for building an ADT 
for the representation and manipulation of polynomials in a single variable (say 
x). Two such polynomials are 


a(x) = 3x? +.2¢-4 and b(x) = x8 - 10c5 - 3x3 +1 


The polynomial « (x) has 3 terms 3x, 2x, and -4. The coefficients of these terms 
are 3, 2, and —4, respectively; their exponents are 2, 1, and 0. A term of a polyno- 
mial may be represented as a (coefficient, exponent) pair. For example, (3, 2) 
represents the term 3x7. A term whose coefficient is nonzero is called a nonzero 
term. Normally, terms with zero coefficients are not displayed. The term with 
exponent equal to zero (e.g., (—4, 0) in a(x)) does not show the variable, since x 
raised to a power of zero is 1. The degree of a polynomial is the largest exponent 
from among the nonzero terms. There are standard mathematical definitions for 
the sum and product of polynomiais. Assume that we have two polynomials, 
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a(x) = Yajx! and B(x) = Ybjx'; then 
a(x) + B(x) = Yq; + b))x! 
a(x): b(x) = L(@; x! - Dib; x/)) 


Similarly, we can define subtraction and division on polynomials, as well as 
many other operations. We begin with an ADT definition (ADT 2.3) of a polyno- 
mial. ADT 2.3 shows only a subset of the functions we may want to perform on 


polynomials. The ADT is easily extended to include functions for the subtract, 
divide, input and output operations. 


class Polynomial { 


H p(x)=agx™ + --+ + a,x"; a set of ordered pairs of <e;, a;>, 


H where a; is a nonzero float coefficient and e; is a non-negative integer exponent. 
public: 


Polynomial(); 

# Construct the polynomial p(x) = 0. 

Polynomial Add(Polynomial poly); 

# Return the sum of the polynomials *this and poly. 

Polynomial Mult(Polynomial poly); 

# Return the product of the polynomials *this and poly. 

float Eval(float f); 

Evaluate the polynomial *this at f and return the result. 
k 


ADT 2.3: Abstract data type Polynomial 


2.3.1 Polynomial Representation 


We are now teady to make some representation decisions. A very reasonable 
first decision would be to arrange the terms in decreasing order of exponent. 


This considerably simplifies many of the operations. We discuss three represen- 
tations that are based on this principle: 


Representation 1: One way to represent polynomials in C++ is to define the 
Private data members of Polynomial as follows: 
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private: 
int degree; Mf degree < MaxDegree 
float coef [MaxDegree + 1]; // coefficient array 


where MaxDegree is a constant that represents the largest-degree polynomial 
that is to be represented. Now, if a is a Polynomial class object and 
n S MaxDegree, then the polynomial a(x) above would be represented as: 


adegree =n 
a.coef[i]=a,-;,0SiSn 


Note that a.cvef[i] is the coefficient of x”~! and the coefficients are stored 
in order of decreasing exponents. This representation leads to very simple algo- 
rithms for many of the operations on polynomials (addition, subtraction, evalua- 
tion, multiplication, etc.). 


Representation 2: Representation | requires us to know the maximum degree of 
the polynomials we expect to work with and also is quite wasteful in its use of 
computer memory. For instance, if a.degree is much less than MaxDegree, then 
most of the positions in the array a.coef {] are unused. We can overcome both of 
these deficiencies by defining coef so that its size is a.degree + 1. This can be 
done by declaring the following private data members 


private: 
int degree; 
float *coef; 


and adding the following constructor to Polynomial: 


Polynomial::Polynomial(int d) 
{ ' 
degree =d; 
coef = new float (degree+}); 


} 


Representation 3: Although Representation 2 solves the problems mentioned 
earlier, it does not yield a desirable representation. To see this, let us consider 
polynomials that have many zero terms. Such polynomials are called sparse. 
For instance, the polynomial x! +1 has two nonzero terms and 999 zero 
terms. Consequently, 999 of the entries in coef will be zero if Representation 2 
is used. To overcome this problem, we store only the nonzero terms. For this 
purpose, we define the class rerm as below. 
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class Polynomial ; // forward declaration 


class Term { 
friend Polynomial; 
private: 
float coef; // coefficient 
intexp; —// exponent 


‘ 


The private data members of Polynomial are defined as follows: 
private: 

Term *termArray; / array of nonzero terms 

int capacity; H size of termArmay 

int terms; / number of nonzero terms 


Before proceeding, we should compare our current representation with 
Representation 2. Representation 3 is definitely superior when there are many 
zero terms. For example, ¢(x) = 2x! + 1 uses only 6 units of space (one for 
c.capacity, one for c.terms, two for the coefficients, and two for the exponents) 
when we use Representation 3 but when Representation 2 is used, 1002 units of 
space are required. However, when all terms are nonzero, as in a(x) above, 
Representation 3 uses about twice as much space as does Representation 2. 
Unless we know beforehand that each of our polynomials has very few zero 
terms, Representation 3 is preferable. The exercises explore an alternative 
representation that uses the STL class vector instead of an array. 


2.3.2 Polynomial Addition 


Let us now write a C++ function to add two polynomials, @ and b, to obtain the 
sum c =a + b. It is assumed that Representation 3, above, is used to store a and 
b, Function Add (Program 2.8) adds a (x) (*this) and (x) term by term to pro- 
duce ¢(x). This function assumes that the default constructor for Polynomial ini- 
tializes capacity and terms to | and 0, respectively, and initializes termArray to 
an array with 1 position (i.¢., the size or capacity of termArray is 1). 

The basic loop of this algorithm consists of merging the terms of the two 
polynomials, depending upon the result of comparing the exponents. The if 
Statement determines how the exponents are related and performs the proper 
action. Since the tests within the while statement require two terms, if one poly- 
nomial runs out of terms, we must exit the loop. The remaining terms of the 
other polynomial can be copied directly into the result. The terms of c are 
entered into its array termArray by calling function NewZerm (Program 2.9). In 
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1 Polynomial Polynomial::Add(Polynomial b) 
2 {(/# Return the sum of of the polynomials *this and b. 


3. Polynomial c; 
4 int aPos =0, bPos = 0; 
5 — while ((aPos < terms) && (bPos < b.terms)) 
6 if ((termArray [aPos ].exp == b.termArray [bPos ).exp) { 
7 float 1 = termArray [aPos |.coef + b.termArray [bPos }.coef; 
8 if (t) c. NewTerm (1, termArray (aPos }.exp); 
9 aPos++; bPos++; 
10 } 
M else if ((termArray [aPos ].exp < b.termArray (bPos }.exp) { 
12 c.NewTerm (b.termArray [bPos }.coef, b.termArray {bPos }.exp); 
13 bPost+; 
14 } 
15 else { 
16 ¢.NewTerm (termArray [aPos ].coef, termArray {aPos }.exp); 
17 aPos++; 
18 


} 
19 // add in remaining terms of *this 
20 ~~ for (; aPos < terms ; aPos++) 
21 c.NewTerm (termArray [aPos }.coef, termArray (aPos }.exp); 
22 =H add in remaining terms of b(x) 
23 ~~ for (; bPos < b.terms; b++) 
24 c.NewTerm (b.termArray [bPos }.coef, b.termArray (bPos ).exp); 
25 returnc; 
26) 


Program 2.8: Adding two polynomials 


case there is not enough space in fermArray to accommodate the new term, its 
capacity is doubled. If we don’t have enough memory to create the array temp, 
new will throw an exception. Since neither NewTerm nor Add catch this excep- 
tion, control is passed to the function that invoked Add whenever new fails. If the 
thrown exception is not caught by any part of the program, the program ter- 
minates. 


Analysis of Add: St is natural to carry out this analysis in terms of the number of 
nonzero terms in a (*this) and 6. Let m and n be the number of nonzero terms in 
a and b respectively. Lines 3 and 4 contribute O(1) to the overall computing 
time. In each iteration of the while loop, aPos or bPos or both increase by 1. 
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yoid Polynomial::NewTerm(const float theCoeff, const int theExp) 
{i Add a new term to the end of termArray. 
if (terms == capacity) 
{/ double capacity of fermArray 
capacity *= 2; 
term *temp = new term capacity}; // new array 
copy(termArray, termArray + terms, temp); 


delete (| termArray; 4 deallocate old memory 
termArray = temp; 


termArray [terms] .coef = theCoeff, 
termArray [terms++].exp = theExp; 
} 


Program 2.9: Adding a new term, doubling array size when necessary 


Since the while loop terminates when either aPos equals a.terms or b.Pos equals 
b.terms, the number of iterations of this loop is bounded by m +n—-1. This 
worst-case is achieved, for instance, when a(x}= >i x and 
b@)=y" x7/+!_ Since none of the exponents are the same in a(x) and b(x), 
termArray |aPos ].exp # termArray [bPos |.exp. Consequently, on each iteration 
the value of only one of aPos or bPos increases by 1. For the moment, let’s 
ignore the time spent doubling the capacity of c.termArray in NewTerm. 
Exclusive of this time, each iteration of the while loop takes O(1) time. So, the 
total time contributed by this loop is O(m + 2). The for loops of lines 20 and 23 
also contribute O(m + n) time to the overall complexity. Adding together the 
time contributions of all components of Add, we obtain O(m +n + time spent in 
array doubling) as the asymptotic computing time of this algorithm. 

Although it may appear that a lot of time is spent doubling the size of ter- 
mArray, this is actually not the case. To create an array of type Term and size s, 
we must allocate sufficient space for s terms and also invoke the constructor for 
Term for each position in the array. Following the creation of the new array, we 
copy elements from the old array into the new one. In all our analyses of array 
doubling, we assume that it takes O(1) time to allocate memory and that each 
invocation of the constructor and each element copy takes O(1) time. Hence the 
time needed to double an array is linear in the size of the new array. Initially, 
¢.capacity is 1. Suppose that when Add terminates, c.capacity is 2* for some k, 


k>0. The total time spent over all array doublings is O( ¥:2') = O(2**!) = O(24). 


i=l 
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Since c.terms > 2*-! (otherwise the array size would not have been doubled from 
2'-! to 24) and m+n > c.terms, the total time spend in array doubling is 
O(c.terms) = O(m +n). Hence even with the time spent on array doubling 
added in, the total run time of Add is O(n + n). 

Array doubling contributes at most a constant multiplicative factor to the 
tun time of Add! This latter statement is true even if we resize using any con- 
stant multiplicative factor greater than 1 (rather than by a factor of 2 as in array 
doubling) but is not true when we resize by an additive constant (i.e., increase 
the size by c, c 2 1, whenever additional space is needed). Experiments indicate 
that array doubling is responsible for a very small fraction of the total run time of 
Add.Q 


Notice that when Add terminates, capacity 2 terms for the result polyno- 
mial. We can recover any excess space in the array termArray by seducing its 
size to equal the number of terms in the polynomial. The code to reduce the size 
of an array is very similar to that used in Program 2.9 to increase array size. This 
reduction in size doesn’t change the asymptotic complexity of Add. 

The exercises examine an alternative array representation of polynomials 
and, in Chapter 4, we develop a linked representation for them. 


EXERCISES 


1. Use the six operations defined in this section for an ordered Jist to arrive at 
an ADT specification for such a list. 

2. Ifa = (ag, ** +, @,-1) and b = (bo, ---, b»-1) are ordered lists, then a <b 
if a = 6; for OSi <j and a; < bj, or, if a; =; for OSi <n and n<m. 
Write a function that returns -1, 0, or +1, depending upon whether 
a<b,a=b,ora>b. Assume that the a;s and bys are integer. 

3. Modify function Add so that it reduces the size of c.termArray to c.terms 
prior to termination. With this modification, can we dispense with the data 
member capacity? 

4. Write C++ functions to input and output polynomials represented as in this 
section. Your functions should overload the << and >> operators. 

5. Write a C++ function that multiplies two polynomials represented as in this 
section. What is the computing time of your function? 

6. Write a C++ function that evaluates a polynomial at a value x9 using the 
representation of this section. Try to minimize the number of operations. 

7. The polynomials a(x) =x eg ext ex? and B(x) = at 
xl 4°. +- 4x34 x cause Add to work very hard. For these polynomi- 
als, determine the exact number of times cach statement will be executed. 
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8. Develop a C++ class Polynomial that uses an STL vector instead of ai 
array to hold the nonzero terms. You must include functions to add sae 
tract and multiply polynomials. Also, you must overload the input and out. 
put operators << and >> so that these work with objects of type Polyno- 
mial, What is the complexity of each function in your class? Compare the 
merits of using a vector over using an array. 


In an alternative representation of polynomials, we store the nonzero terms 
of all polynomials in a single array called termArray. Each element in ter- 
mArray is of type Term, which we defined in the section. Since the Single 
array termArray is to be shared by all Polynomial objects, we declare it as 
a Static class data member of Polynomial. The private data members of 
Polynomial are defined as follows: 
private: 

static Term *rermArray; 

static int capacity; 

Static int free ; 

int start, finish; 


where capacity is the size of the array termArray. The required definitions 
of the static class members outside the class definition are: 

int Polynomial::capacity = 100; 

Term Polynomial::termArray = new Term[{ 100); 

int Polynomial::free = 0; // \ocation of next free location in termArray 


Consider the two polynomials a(x) = 2x 100 4) and b@)= 
x44 10x? + 3x +1. These could be stored in the array termArray as 
shown in Figure 2.1. Note that a.start and b.start give the location of the 
first term of a and b respectively, whereas a.finish and b.finish give the 
location of the last term of a and b. The static class member free gives the 
location of the next free location in the array termArray. For our example, 
astart = 0, a.finish = 1, b.start = 2, b.finish = 5, and free = 6. 

In general, any polynomial a@ that has n nonzero terms has a.start 
and a.finish such that a.finish = a.start +n — 1. If a has no nonzero terms 
{i.e.,@ is the zero polynomial), then a.finish = a.start — 1. 

Write and test C++ functions to input, output, add, multiply, and 
evaluate polynomials represented in this way. Use array doubling to 
increase the size af termArray whenever needed. Is the representation of 
this exercise better or worse than the representation used in the text? 
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astart a.finish b.start b.finish free 
4 L 4 L L 
coef 2. | 1 | 1 10 3 1 
exp 1000 | 0 | 4 3 2 0 
0 1 2 3 4 5 6 


Figure 2.1: Array representation of two polynomials 


10. Show that if we start with an array whose size is 10 and increase its size by 
a constant multiplicative factor c, ¢ > 1, whenever we need additional 
space, the total time spent in all array resizings is linear in the final size of 
the array. (Assume that the size of the new array is obtained by rounding 
up the product of ¢ and the initial array size, if this quantity is not a whole 
number.) 


11. Show that if we start with an array whose size is I and increase its size by a 
constant additive factor c, c 2 1, whenever we need additional space, the 
total time spent in all array resizings is quadratic in the final size of the 
array. 


2.4 SPARSE MATRICES 


2.4.1 Introduction 


A matrix is a mathematical object that arises in many physical problems. As 
computer scientists, we are interested in studying ways to represent matrices so 
that the operations to be performed on them can be carried out efficiently. A gen- 
eral matrix consists of m rows and n columns of numbers, as in Figure 2.2. The 
first matrix has five rows and three columns, the second six rows and six 
columns. In general, we write m x n (read *‘m by n’’) to designate a matrix with 
m rows and ” columns. Such a matrix has ma elements. When m is equal to n, 
we call the matrix square. 

It is very natural to store a matrix in a two-dimensional array, say a[m ][n]. 
Then we can work with any element by writing a [i ][j], and this element can be 
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col 0 col 1 col 2 col 0 col 1 col 2 col 3 col 4 col 5 

row0 |-27 3 4 rowOlIS O O 22 0 -15 
tow | 6 82 -2 row!| 0 Il 0 0 0 
row2 |109 -64 11 row210 0 0 6 0 QO 
row3 | 12 8 9 row3} 0 0 0 0 0 O 
row4 | 48 27 47 row 4/91 0 0 0 0 0 

row5) 0 0 28 0 0 0 

{a) (b) 


Figure 2.2: Two matrices 


found very quickly, as we will see in the next section. Now if we look at the 
second matrix of Figure 2.2, we see that it has many zero entries. Such a matrix 
is called sparse. There is no precise definition of when a matrix is sparse and 
when it is not, but it is a concept that we can all recognize intuitively. In the 
matrix of Figure 2.2(b), only eight out of 36 possible elements are nonzero, and 
that is sparse! A sparse matrix requires us to consider an alternative form of 
representation. This comes about because in practice many of the matrices we 
want to deal with are large, e.g., 5000 x 5000, but at the same time they are 
sparse: say only 5000 out of 25 million possible elements are nonzero. When a 
5000 x 5000 array is used to store this matrix, we need 25 million units of space. 
Also, 25 million units of time are needed for operations such as addition and 
transposition. Using an alternative representation, which stores only the nonzero 
elements, we can reduce both the space and time requirements considerably. 
Before developing a particular representation for sparse matrices, we first 
must consider the operations that we want to perform on these matrices. A 
minimal set of operations includes matrix creation, transposition, addition, and 
multiplication. ADT 2.4 contains our specification of the matrix ADT. 
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class SparseMatrix 
{/ A sct of triples, <row, column, value>, where row and column are non-negative 
4 integers and form a unique combination; value is also an integer. 
public: 
SparseMatrix(int r, int c, int); 
/ The constructor function creates a SparseMatrix with 
Hr sows, c columns, and a capacity of ¢ nonzero terms. 


SparseMatrix Transpose(); 
4 Returns the SparseMatrix obtained by interchanging the row and column 
4 value of every tiple in *this. 


SparseMatrix Add(SparseMatrix b); 

//\€ the dimensions of *this and b are the same, then the matrix produced by 
H adding corresponding items, namely those with identical row and column 
4 values is returned; otherwise, an exception is thrown. 


SparseMatrix Multiply(SparseMatrix b); 

4 Xf the number of columns in *this equals the number of rows in b then the 

H matrix d produced by multiplying this and 5 according to the formula 

Hd (iG = D@ LE Mk] - 6 (KIL), where 4 [iL] is the (, f)th element, is returned 
Hf k ranges from 0 to one less than the number of columns in *this; 

4 otherwise, an exception is thrown. 


HH 
ADT 2.4: Abstract data type SparseMatrix 


2.4.2 Sparse Matrix Representation 


Before implementing any of these operations, we must establish the representa- 
tion of the sparse matrix. By examining Figure 2.2, we know that we can charac- 
terize uniquely any element within a matrix by using the triple 
<row, col, value >. This means that we can use an array of triples to represent a 
sparse matrix. We require that these triples be stored by rows with the triples for 
the first row first, followed by those of the second row, and so on. We also 
require that all the triples for any row be stored so that the column indices are in 
ascending order. In addition, to ensure that the operations terminate, we must 
know the sumber of rows and columns and the number of nonzero elements in 
the matrix. Putting all this information together suggests that we define: 
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class SparseMatrix ; if forward declaration 


class MatrixTerm { 
friend class SparseMatrix 
private: 

int row, col, value; 
% 


and in class SparseMatrix: 


private: 
int rows, cols, terms, capacity; 
MatrixTerm *smArray; 


where rows is the number of rows in the matrix; cols is the number of columns; 
terms is the total number of nonzero entries; and capacity is the size of smArra . 
Figure 2.3(a) shows the representation of the matrix of Figure 2.2(b) using awiaee 
ray. Positions 0 through 7 of smArray store the triples representing the nonzero 
entries. The triples are ordered by row and within rows by columns. 


(1) 
{2] 
(3] 
2) 
{5] 
(6] 
7] 


row col value 
smArray[0) 0 0 15 smArray[0) 
(i) i) 3 22 
[2] 0 § -15 
(3) 1 1 uy 
[4] 1 2 3 
{3} 2 3 + 
{6] 4 0 91 
07] 5 2 28 
(a) 


row 


vN-oOO} 


SK Uwwn 


Ss 


col 


ONnourH sO 


Figure 2.3: Sparse matrix and its transpose stored as triples 
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2.4.3 Transposing a Matrix 


Figure 2.3(b) shows the transpose of the matrix of Figure 2,3(a). To transpose a 
matrix, we must interchange the rows and columns. This means that if an ele- 
ment is at position [#][/] in the original matrix, then it is at position [j)[i] in the 
transposed matrix. When i=j, the elements on the diagonal will remain 
unchanged. Since the original matrix is organized by rows, our first idea for a 
transpose algorithm might be the following: 


for (each row t) 
store (i,j,value) of the original matrix as (j,i, value) of the wanspose; 


The difficulty is in not knowing where to put the element (j,i,vaiue) until all 
other elements that precede it have been processed. In Figure 2.3(a), for 
instance, we encounter 


(0,0, 15) which becomes (0,0, 15) 
(0,3,22) | which becomes (3, 0, 22) 
(0,5,-15) which becomes (5, 0,-15) 
dd, 1,11) which becomes (1, 1,11) 


We can avoid this difficulty of not knowing where to place an element by chang- 
ing the order in which we place elements into the transpose. Consider the fol- 
lowing strategy. 


for (all elements in column j) 
store (i,j,value) of the original matrix as (j,i,value) of the transpose; 


This says ‘‘find all elements in column 0 and store them in row 0, find all ele- 
ments in column | and store them in row 1, etc.”’ Since the rows are originally in 
order, this means that we will locate elements in the correct column order as 
well. The function Transpose (Program 2.10) computes and returns the transpose 
of +this. 

It is not too difficult to see that the function is correct. The variable 
currentB always gives us the position in b where the next term in the transpose is 
to be inserted. The terms in b.smArray are generated by rows. Since the rows of 
b are the columns of *this, row c of b is obtained by collecting all the nonzero 
terms in column c of *this. This is precisely what is being done in lines 7 
through 15. In the first iteration of the for loop of lines 7 through 15, all terms 
from column 0 of *this are moved to b; in the next iteration, all terms from 
column | are moved; and so on. 


Analysis of Transpose: Let cols, rows, and terms denote the number of 
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1 SparseMatrix SparseMatrix::Transpose( y 
2 {é/ Return the transpose of *this. ; 
3. SparseMatrix b{cols, rows, terms); Hf capacity of b.smArray is terms 
4 if (terms > 0) 

5 {4 nonzero matrix 

6 int currentB =0; 

a for (int c = 0; ¢ < cols ; c++) // transpose by columns 

8 for (int i=03i< terms 3 i++) 

9 # find and move terms in column c 


10 if (smArray [i).col == c) 

YW { 

12 b.smArray [currentB ).row = c3 

13 b.smArray [currentB }.col = smArray [i }.row; 
14 


b.smArray [currentB ++].value = smArray {i ].value; 


} 
16 } 4 end of if (terms > 0) 
7 return 5; 


Program 2.10: Transposing a matrix 


columns, rows, and terms, respectively, in *this. For each iteration of the loop of 
lines 7 through 15, the condition of line 10 is tested terms times. Since the 
number of iterations of this loop is cols, the total time for line 10 is terms «cols. 
The assignments of lines 12-14 take place exactly terms times, since there are 
only this many nonzero terms in the sparse matrix being generated. Lines 3-6 
take a constant amount of time. The total time for Transpose is therefore 
Otterms -cols). In addition to the space needed for *this and b, the function 


requires only a constant amount of space, i.e., space for the variables c, i, and 
currentB. O 


We now have a matrix transpose algorithm that we believe is correct and 
that has a computing time of O(terms -cols). This computing time is a little dis- 
turbing, since we know that in case the matrices had been Tepresented as two- 
dimensional arrays, we could have obtained the transpose of a rows x cols matrix 
in time O(rows -cols). The algorithm for this has the simple form 
for (int | = 05 j< cols; j++) 

for (int § = 0 ;i< rows pitt) 


bY MA = oll: 
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The O(terms - cots) time for function transpose becomes O(rows- cols?) 
when terms is of the order of rows:cols. This is worse than the O(rows-cols) 
time using the simple form. Perhaps, in an effort to conserve space, we have 
traded away too much time. Actually, we can transpose a matrix represented as 
a sequence of triples in time O(terms + cols) by using a little more space. The 
new algorithm, FastTranspose (Program 2.11), proceeds by first determining the 
number of elements in each column of *this, This gives us the number of ele- 
ments in each row of &. From this information, the starting point in of each of 
its rows is easily obtained. We can now move the elements of *this one by one 
into their correct position in b. 

The correctness of function FastTranspose follows from the preceding dis- 
cussion and the observation that the starting point, rowStart [i], of row i, i > 0, of 
b is rowStart [i — 1] + rowSize [i ~ 1], where rowSize [i — 1] is the number of ele- 
ments in row i — | of b. The computation of rowSize and rowStart is carried out 
in lines 9-13. In lines 14-21 the elements of *this are placed one by one into the 
correct place in b. rowStart ([] is maintained so that it is always the position in b 
where the next element in row j of 0 is to be inserted. If we try the algorithm on 
the sparse matrix of Figure 2.3(a), then after the execution of fine 13, the values 
of rowSize and rowStart are: 


) ) 2) 8) 4) [5) 
rowSize = 3 2 1 0 1 1 
rowStart= 0 3 5 6 6 7 


There are three loops in FastTranspose, which are executed terms, cols—1, 
and terms times respectively. Each iteration of each loop takes a constant 
amount of time, so the time for all 3 loops is O(cols + terms). Line 9 takes 
O(cols) time and the remaining lines take O(1) time. So, the overall complexity 
is O{cols + terms). 

The computing time of O(cols + terms) becomes O(rows - cols) when terms 
is of the order of rows -cols. This is the same as when two-dimensional arrays 
were in use. However, the constant factor associated with FastTranspose is 
larger than that for the array algorithm. When terms is sufficiently small com- 
pared to its maximum of rows - cols, FastTranspose will be faster. Hence in this 
representation, we save both space and time! This was not true of Transpose, 
since terms will almost always be greater than max{rows, cols} and cols: terms 
will therefore always be at least rows - cols. The constant factor associated with 
Transpose is also larger than the one in the array algorithm. Finally, you should 
note that FastTranspose requires more space than does Transpose. The space 
required by FastTranspose can be reduced by utilizing the same space to 
represent the two arrays rowSize and rowStart. 
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1 SparseMatrix SparseMatrix::FastTranspose() 
2. {i Retum the transpose of *this in Otterms +cols) time. 
SparseMatrix b(cols, rows, terms 3 
if (rerms > 0) 
{// nonzero matrix 
int *rowSize = new int[cols]; 
int *rowStart = new int{cols]; 
8 H compute rowSize [i] = number of terms in row i of b 
9 fill(rowSize, rowSize + cols, 0); # initialize 
0 for (i= 03 i < terms ; i++) rowSize[smArray(i].col}++ 5 


BAWEY 


cut i rowStart (i) = starting position of row i in b 
12 rowStart{0] = 0; 


13 for (i= 1 5 i < cols ; i++) rowStart{i] = rowStart{i— 1] + rowSize[i-\); 
14 for (i =0 5 i< terms ; i++) 

15 {// copy from *this to b 

16 int j = rowStart(smArray(i).col}; 

7 b.smArray|j].row = smArray(i).col; 

18 b.smArray|j).col = smArray(i).row 5 

9 b.smArray|j].value = smArray|i).value ; 

20 rowStart{smArray{i).col}++ 5 


21 } end of for 

2 delete [} rowSize ; 
23 delete [] rowStart ; 
24} Mend of if 

25 return b; 


Program 2.11: Transposing a matrix faster 


2.4.4 Matrix Multiplication 


Definition: Given a and b, where a is m xn and b is n x p, the product matrix d 
has dimension m x p. Its [i] [j] element is 
nzl 


dij = Lain byy 
5 * &=0 
forOsi<mandQ<j<p.0 


The product of two sparse matrices may no longer be sparse, as Figure 2.4 
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shows. 


100] flit 11t 
100) |000;=]111 
100} [000 11t 


Figure 2.4: Multiplication of two sparse matrices 


We would like to multiply two sparse matrices a and b represented as 
ordered lists as in Figure 2.3. We need to compute the elements of d by rows so 
that we can store them in their proper place without moving previously com- 
puted elements. To do this we pick a row of a and find all elements in column j 
of b for j = 0, I, -**, b.cols~1, where b.cols is the number of columns in b. 
Normally, we would have to scan all of to find alli the elements in column j. 
However, we can avoid this by first computing the transpose of b. This puts all 
column elements in consecutive order. Once we have located the elements of 
row i of A and column j of b, we just do a merge operation similar to that used 
for polynomial addition in Section 2.3. An alternative approach is explored in 
the exercises. 

Before we write a matrix multiplication function, it will be useful to define 
the function of Program 2.12, which stores a matrix term. This function uses 
another function ChangeSize tD (Program 2.13), which changes the size of a I- 
dimensional array. 


void SparseMatrix::StoreSum (const int sum, const int r, const int c) 
{// If sum != 0, then it along with its row and column position are stored as the 
# last term in *this. 
if (sum != 0) { 
if (terms == capacity) 
ChangeSize 1D (2*capacity); {1 double size 

smArray(terms).row = r 5 
smArray{terms].col =c¢ 5 
smArray[terms++].value = sum; 


} 


Program 2.12: Storing a matrix term 


The function Multiply (Program 2.14) multiplies the sparse matrices a 
(this) and 6 to obtain the product matrix d using the strategy outlined above. 
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void ChangeSize!D(const int newSize) 

(/ Change the size of smArray to newSize. 
if (newSize < terms) throw “New size must be >= number of terms"; 
MatrixTerm *temp = new MatrixTerm{newSize]; / new array 
copy(smArray, smArray + terms, temp); 
delete [| smArray; if deallocate old memory 
smArray = temp; 
capacity = newSize; 


} 


Program 2.13; Change the size of a 1-dimensional array 


Function Multiply (which is invoked as a.Multiply(b)) makes use of variables 
currRowindex, currColindex, currRowA, currColB, and currRowBegin. The 
variable currRowA is the row of a that is currently being multiplied with the 
columns of b. currRowBegin is the position in a of the first element of row 
currRowA, and currColB is the column of 6 that is currently being multiplied 
with row currRowA of a. The variables currRowIndex and currColIndex are 
used to examine successive elements of row currRowA and column currColB of 
aand b, respectively. If matrices a and b are incompatible, then Multiply throws 
an exception in Line 3. Lines 10-14 of the function introduce a dummy term into 
each of a and bXpose. This enables us to handle end conditions (i.c., computa- 
tions involving the last row of a or last column of b) in an elegant way. Notice 
that because of the use of array doubling in function StoreSum, the capacity of 
d.smArray may exceed the number of terms in the matrix. Also, because of the 
addition of the dummy term to *this its capacity will be more than the number of 
terms in *this. We can free the unused space in these two matrices by using the 
function ChangeSize 1D. 


| SparseMatrix SparseMatrix::Multiply(SparseMatrix b) 
2 {# Return the product of the Sparse matrices *this and b. 
3. if (cols != b.rows) throw “Incompatible matrices"; 

4 SparseMatrix bXpose = b.FastTranspose(); 

5 SparseMatrix d(rows, b.cols, 0); 

6 int currRowindex = 0, 

7 currRowBegin =0, 

8 currRowA = smArray(0].row; 

9H set boundary conditions 


10 if (terms == capacity) ChangeSize!D(terms + 1); 
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11 bXpose. ChangeSize 1 D(bXpose.terms + 1); 
12. smArray[terms).row = rows; 

13. bXpose.smArray[b.terms}.row = b.cols; 

14 bXpose.smArray[b.terms).col =—1 5 

15 int sum =0; 

16 while (currRowIndex < terms) 

17 {/ generate row currentRowA of d 

18 int currColB = bXpose.smArray[0).row; 
19 int currColindex =0; 

20 while (currColindex <= b.terms) 

21 {// multiply row currRowA of *this by column currColB of b 


22 if (smArray(currRowIndex].row '= currRowA) 

23 {// end of row currRowA 

24 d.StoreSum(sum, currRowA, currColB); 

25 sum=0; / reset sum 

26 currRowIndex = currRowBegin; 

27 / advance to next column 

28 while (bXpose.smArray[currColindex].row == currColB) 
29 currColindex++; 

30 currColB = bXpose.smArray{currColindex].row; 

31 


} 
32 else if (bXpose.smArray|currColindex).row != currColB) 
33 {// end of column currColB of b 


34 d.StoreSum(sum, currRowA, currColB); 

35 sum=0; HH reset sum 

36 H set to multiply row currRowA with next column 
37 currRowIndex = currRowBegin; 

38 currColB = bXpose.smArray(currColIndex).rows 
39 } 

40 else 

41 if (smArray{currRowindex}.col < 

42 bXpose.smArray(currColindex}.col) 

43 currRowIndex-++ ;_ if advance to next term in row 
44 else if (senArray[currRow!Index].col == 

45 bXpose.smArray|currColindex].col) 

46 {# add to sum 

47 sum += smArray[currRowIndex}.value * 

48 bXpose.smArray|currColindex].value; 
49 currRowindex++; currColindex++; 

50 

51 else currColindex++; // next term in currColB 


52 } # end of while (currColindex <= b.terms) 
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53 while (smArraylcurrRowindex |.row == currRowA) ff advance to next row 
S4 currRowindextt; 

5S currRowBegin = currRowIndex; 

56 currRowA = smArray(currRowIndex].row; 


57} fend of while (currRowindex < terms) 
58 return d; 
59) 


Program 2.14: Multiplying sparse matrices 


Analysis of Multiply: We leave the correctness proof of this algorithm as an 
exercise. Let us examine its computing time. In addition to the space needed for 
a (this), b, d, and some simple variables, space is needed for the transpose 
matrix bXpose. Algorithm FastTranspose also needs some additional space. The 
exercises explore a strategy for Multiply that does not explicitly compute 
bXpose, and the only additional space needed is the same as that required by 
FastTranspose. Turning our attention to the computing time of Multiply, we see 
that lines 3-15 require only O(b.cols + b.terms) time. The while loop of lines 
16-57 is executed at most a.rows times (once for each row of a). In each itera- 
tion of the while loop of lines 20-52, the value of currRowIndex or currColIndex 
or both increases by 1, or currRowindex and currColB are reset. The maximum 
total increment in currColindex over the whole loop is b.terms. If 1, is the 
number of terms in row r of a, then the value of currRowIndex can increase at 
most f, times before currRowindex moves to the next row of a. When this hap- 
pens, currRowindex is reset to currRowBegin in line 26. At the same time 
currColB is advanced to the next column. Hence, this resetting can take place at 
most b.cols times. The total maximum increment in currRowindex is therefore 
b.cols -1,, The maximum number of iterations of the while loop of lines 20-52 is 
therefore b.cols + b.cols +t, + b.terms. The time for this loop when multiplying 
with row r of a is O(b.cols «t, + b.terms). Lines 53-54 take only O(f,) time. 
Hence, the time for the outer while loop, lines 16-57, for the iteration with row 
currRowA of a, is O(b.cols «1, + b.terms). The overall time for this loop is then 
O(E, (b.cols +t, + b.terms)) = O(b.cols- a.terms + a.rows + b.terms). 

Once again, we may compare the computing time with the time to multiply 
matrices when arrays are used. The classical multiplication algorithmn is: 
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for (int i = 0; i < a.rows; i++) 
for (int j =0; j< b.cols; j++) 


sum = 0; 

for (int k = 0; k < a.cols; k++) 
sum += a [i J{k} * O(4)U1; 

¢[#)))] = sum; 


The time for this is O(a.rows -a.cols ‘ b.cols). Since 
a.terms $ a.rows -a.cols and b.terms S$ a.cols - b.rows, the time for Multiply is at 
most O(a.rows -a.cols -b.cols). However, its constant factor is greater than that 
for matrix multiplication using arrays. In the worst case, when 
a.terms = a.rows-a.cols or b.terms = a.cols - b.rows, Multiply will be slower by 
a constant factor. However, when a.terms and b.terms are sufficiently smaller 
than their maximum values, i.e., a and b are sparse, Multiply will outperform the 
above multiplication algorithm for arrays. 

The above analysis for Multiply is nontrivial. It introduces some new con- 
cepts in algorithm analysis and you should make sure you understand the 
analysis. O 


EXERCISES 


1. How much time does it take to locate an arbitrary element A {i JL] in the 
representation of this section and to change its value? 


2. Analyze carefully the computing time and storage requirements of function 
FastTranspose (Program 2.11). What can you say about the existence of an 
even faster algorithm? 


3. Write C++ functions to input and output a sparse matrix. These should be 
implemented by overloading the >> and << operators. You should design 
the input and output formats. However, the internal representation should 
be a one dimensional array of nonzero terms as used in this section. 
Analyze the computing time of your functions. 


4, Rewrite function FastTranspose (Program 2.11) so that it uses only one 
array rather than the two arrays required to hold RowSize and RowStart. 

5. Develop a correctness proof for function Multiply (Program 2.14). 
Use the row pointers idea used in FastTranspose and rewrite Multiply (Pro- 
gram 2.14) to multiply two sparse matrices represented as in Section 2.4 


without explicitly transposing either. What is the computing time of your 
algorithm? 
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7. A variation of the scheme discussed in this section for sparse matrix 
representation involves representing only the nonzero terms in a one. 
dimensional array v in the order described. In addition, a strip of nx m 
bits, bits{a][m] is also kept. dits[i 1 = 0 if aff JU] = 0, and bits(i Ij] = 1 
if a{i]{j] #0. The figure below illustrates the representation for the sparse 
matrix of Figure 2.2(b). 


(a) Onacomputer with w bits per word, how much storage is needed to 
Tepresent an n X m sparse matrix that has 1 nonzero terms? 


{b) Write an algorithm to add two sparse matrices represented as above. 
How much time does your algorithm take? 


Discuss the merits of this representation versus the representation of 
Section 2.4. Consider space and time requirements for such opera- 
tions as random access, add, multiply, and transpose. Note that the 
random access time can be improved somewhat by keeping another 


array ra such that ra{i]= number of nonzero terms in rows 0 
through i - 1. 


(c) 


2.8 REPRESENTATION OF ARRAYS 


Multidimensional arrays are usually implemented by storing the elements in a 
‘one-dimensional array. In this section, we develop a representation in which an 
arbitrary array element, say afi }{i2)....,{i,], gets mapped onto a position in a 
one-dimensional C++ array so that it can be retrieved efficiently. This is neces- 
sary since programs using arrays may, in general, use array elements in a random 
order. In addition to being able to retrieve array elements easily, it is also neces- 
Sary to be able to determine the amount of memory to be reserved for a particular 
array. Assuming that each array element requires only one word of memory, the 
number of words needed is the number of elements in the array. If an array is 
Geclared alu }[w2}. ---,[4,], where 0 through u;-1 is the range of index values 
in dimension i, then it is easy to see that the number of elements is 
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One of the common ways to represent an array is in row major order (see 
Exercise 3 for column major order). If we have the declaration 


a{2)[3](23(2} 


then we have a total of 2*#34#2*2 = 24 elements. Using row major order, these 
elements will be stored as 


a{O}[0}{O}[0}. a[O}O}{O}[1}, afO}(O}{11(0}, afONO1IY 
and continuing 

aq{O}(1)0)0}, afO]L) LOVED, af0)(1}(1)(0}, af0I NTI) 
for three more sets of four until we get 

af 1)f21(0}10), af 1FEZIOIAT, af 1)2I110), af 210 


We see that the index at the right moves the fastest. In fact, if we view the 
indices as numbers, we see that they are, in some sense, increasing: 


0000, 0001, --:, 1210, 1211 


A synonym for row major order is lexicographic order. 

The problem is how to translate from the name a[i,Jfi2], -... i,] to the 
correct location in the one-dimensional array. Suppose a[0}[G](0][O] is stored at 
position 0. Then a[0][0}[0}[1] will be at position 1 and a{1)[2}[1}{1] at position 
23. These two addresses are easy to guess. In general, we can derive a formula 
for the address of any element. This formula makes use of only the starting 
address of the array plus the declared dimensions. 

Before obtaining a formula for the case of an n-dimensional array, let us 
look at the row major representation of one-, two-, and three-dimensional arrays. 
To begin with, if a is declared a{u,], then assuming one word per element, it 
may be represented in sequential memory as in Figure 2.5. If a is the address of 
a{0}, then the address of an arbitrary element a [i] is just a + i. 

The two-dimensional array a[u,]{u2} may be interpreted as u, rows, 
rowg, row), *--, row,,;, each row consisting of i> elements. In a row major 
representation, these rows would be represented in memory as in Figure 2.6. 

Again, if a is the address of a[0)[0], then the address of a[i}[0] is 
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arrayelement: (0) aft} a2) --* ali) ata 
address: a a+] a+2 eis ati see Oty) 


Figure 2.5: Sequential representation of a [1 i] 


+ Wp Wn 
‘elements}elements, 


' rowO row) 


' rowi row uj—1 
' : 


<— i*uyelements —= 
' 1 


(b) 


Figure 2.6: Sequential representation of a [uy ][u2] 


& + i * 2, as there are i rows, each of size uz, preceding the first element in the 
ith row. Knowing the address of a[i}[0], we can say that the address of a{i]l/] 
is then simply & + i * uy + j. 

Figure 2.7 shows the representation of the three-dimensional array 
a) ][¥2][«3]. This array is interpreted as u, two-dimensional arrays of dimen- 
Sion tz X43. To locate afi] ]{k], we first obtain a + iu2u4 as the address for 
4 {F\{O]10] since there are i two-dimensional arrays of size uz X u3 preceding this 
element. From this and the formula for addressing a two-dimensional array, We 
obtain @ + ittquy + jury + kas the address of a[i[j Ik]. 

Generalizing on the preceding discussion, the addressing formula for any 
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eo 


(a) 3-dimensional array A [u ;]{u2][u3] regarded as u, 2-dimensional arrays 


. i at] 


A(0,u2,u3) A(1,u2,u3) A(i,u2,u3) A(u\~1,u2.43) 


<_- iu; elements 


(b) Sequential row major representation of a 3-dimensional array. Each 2- 
dimensional array is represented as in Figure 2.6 


Figure 2.7: Sequential representation of a[w,}[u2}[u3] 


element a[i,J[i2], ---.{i,] in an n-dimensional array declared as 
a(u,)[u2], -**,{4,] may be easily obtained. If a is the address for 
a[0} (0) ---, [0] then &+i,42u3 --* u, is the address for a[i,} [0], --+, [0]. 
The address for afi,][i2](0]---.(0] is then & +i,u2uU3 °°- uy 
+ iQU3ug + 7+ Uy. 

Repeating in this way, the address for a[i,]} [i2], ---, Li,] is 
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K+ iyugts oo Uy 

+ igitgug °° 7 Un 

+ igugus °° Un 

+ inj Un 

+i 

a 
, a> HL we lsj<n 

= : =j+ 
=a+ ¥ ia; where 


a,=1 
jel mi 


Note that a; may be computed from a;,1, 1 Sj <2, using only one multiptica- 
tion as a; = 4) 414)41. Thus, a compiler will initially take the declared bounds 


uy, ‘'*,M, and use them to compute the constants a), -~ 


+, @y_4 using n-2 


multiplications. The address of a[i,], «-, {,] can then be found using the for- 
mula, requiring 2 ~ 1 more multiplications and n additions and n subtractions. 


EXERCISES 


1, {Programming Project} Even though the multidimensional array is pro- 
vided as a standard data object in C+, it is often useful to define your own 
class for multidimensional arrays. This gives a more robust class that: 


(a) 
(b) 


Performs range checking. 


Does not require the index set in each dimension to consist of con- 
secutive integers starting at 0. 


Allows array assignment. 

Allows initialization of an array with another array. 

Selects the range in each dimension of the array during runtime. 
Allows dynamic modification of the range and size of the array. 
Provides a mechanism to determine the size of an array. 


Implement a class mdArray that stores floating point elements and provides 
the functionality specified above. You will need to define two pointer data 
members corresponding to one-dimensional arrays, which will be dynami- 
cally created: the array of elements, and an array that stores the upper and 
lower bounds of each dimension. Additional data members may be 
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required. The data members of your class must be initialized by a con- 
structor that takes as input the number of dimensions and the range of each 
dimension. Also, provide operations to read/write array elements from/to a 
file. 


2. How many values can be held by each of the arrays a{n], &[n){m), 
¢ [a Jf2}(3]? 

3. Obtain an addressing formula for the element a[i,][é2}, ++-, (in) in an 
array declared as a[w}{u2], ---,{u,]. Assume a column-major represen- 
tation of the array with one word per element and @ the address of 
a [0] {0], ..., [0]. In a column-major representation, a two-dimensional array 
is stored sequentially by columns rather than by rows. 


2.6 THE STRING ABSTRACT DATA TYPE 


In this section, we turn our attention to a data type, the string, whose component 
elements are characters. As an ADT, we define a string to have the form S = 
So. °'*, Sa-1, Where s; are characters taken from the character set of the pro- 
gramming language, and n is the length of the string. If 2 = 0, then S is an empty 
or null string. 

There are several useful operations we could specify for strings. Some of 
these operations are similar to those required for other ADTs: creating a new 
empty string, reading a string or printing it out, appending two strings together 
(called concatenation), or copying a string. However, there are other operations 
that are unique to our new ADT, including comparing strings, inserting a sub- 
string into a string, removing a substring from a string, or finding a pattern in a 
string. We have listed some of the essential operations in ADT 2.5, which con- 
tains our specification of the string ADT. 

C++ includes a string class that provides many more functions than are 
specified in our ADT. We do not go into the details of this C++ class here. 
Instead, we focus on the problem of string pattern matching (i.e., the function 
Find specified in ADT 2.5). In the remainder of this section, we assume that, in 
our string class, strings are represented by the private data member str of type 
char*, We may access the ith character of str using either of the notations str {i] 
and *(str + i) and we may assign a string to str using a statement such as str = 
"abe". 
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class String 
{ 
public: 
String(char “init, int m); 
# Constructor that initializes *this to string init of length m, 
pool operator==(String 1); 
#Mf the string represented by *this equals ¢, return true; 
Helse return false. 
bool operator!(); 
Af *this is empty then retum true; else return false. 
int Length(); 
# Return the number of characters in *this. 
String Concat(String t); 
i Return a string whose elements are those of *this followed by those of t, 
String Substr(int i, int j); 
# Retum a string containing the j characters of *this at positions i, i+1, ..., 
Hi + j — 1 if these are valid positions of *this; otherwise, throw an exception, 
int Find(String pat); 
# Return an index i such that par matches the substring of *this that begins 
at position i. Return -1 if pat is either empty or not a substring of *this. 
'; 


ADT 2.5: Abstract data type String 
2.6.1 String Pattern Matching: A Simple Algorithm 


Assume that we have two strings, s and pat, where pat is a pattern to be searched 
for in s. We will determine if pat is in s by using the function Find. The invoca- 
tion s.Find (pat) returns an index i such that pat matches the substring of s that 
begins at position i. It returns -1 if and only if par is either empty or is not a sub- 
string of s. Let us examine how a function Find may be implemented. 

The easiest but least efficient method to determine whether pat is in s is 10 
serially consider each position of s and determine if this position is the starting 
point of a match. Let lengthP and lengthS denote the lengths of the pattern pat 
and the string s, respectively. Positions of s» to the right of position 
lengthS ~ lengthP need not be considered, as there are not enough characters to 
their right to complete a match with pat. Function Find (Program 2.15) 
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implements this strategy. The complexity of this function is 
OlengthP - lengthS). 


int String::Find(String pat) 
{// Return 1 if pat does not occur in *this; 
#/ otherwise return the first position in *this, where pat begins. 
for (int start = 0; start <= Length() — pat.Length0); start++) 
{// check for match beginning at str [start] 


j<pat.Length() && str(start + j) == pat.str[j]; j++) 
pat.Length()) return start; // match found 
‘4 no match at position start 


return ~1 ; // pat is empty or does not occur in s 


} 


Program 2.15: Exhaustive pattern matching 


We can introduce heuristics into function Find that improve its perfor- 
mance on certain pairs of s and pat. For example, for each position start of s 
considered in function Find, we may check for a match of the last character of 
pat with the character at position start + lengthP — | of s before examining char- 
acters 0 through lengthP — 2 of pat for a match. The asymptotic complexity of 
the resulting pattern matching function is still O(lengthP - lengthS). 


2.6.2 String Pattern Matching: The Knuth-Morris-Pratt Algorithm 


Ideally, we would like an algorithm that works in O(lengthP+ lengthS) time. 
This is optimal for this problem, as in the worst case it is necessary to look at all 
characters in the pattern and string at least once. We want to search the string for 
the pattern without moving backwards in the string. That is, if a mismatch 
occurs we want to use our knowledge of the characters in the pattern and the 
position in the pattern, where the mismatch occurred to determine where we 
should continue the search. Knuth, Morris, and Pratt have developed a pattern- 
matching algorithm that works in this way and has linear complexity. Using 
their example, suppose 


pat=abcabcacab 


Let s = 595, °** Sp-; be the string and assume that we are currently 
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determining whether or not there is a match beginning at 5). If s; 


“ ‘ime nt #4, then 
clearly we may proceed by comparing s;,; and a. Similarly, if s, 


: aa 
Sia, #D, then we may proceed by comparing s;,,; and a. if ssa 2 ze ss 
Sj42 #¢ then we have the situation 
s= - ab??? . 2 
pat = @ 8) ¢ @ & tte 


a c¢ ab 
‘The ? implies that we do not know what the character in s is. The first ? in 5 
represents 5; 49, where 5;42 #¢. At this point we know that we may continue the 
search for a match by comparing the first character in pat with s;,2. There is no 
need to compare this character of par with s;,,, since we already know that Siat 
is the same as the second character of pat, b, and so s;,; #4. Let us try this 


again assuming a match of the first four characters in pat followed by a non. 
match, i.e., 5;44 #5. We now have the situation 


s 


e -a@ 3k & 
pat 


b ? 
abcabee 


wa 
a 


re ee: 

aca b 

We observe that the search for a match can proceed by comparing s;,4 and the 
second character in pat, b. This is the first place a partial match can occur by 
sliding the pattern pat towards the right. Thus, by knowing the characters in the 
pattern and the position in the pattern where a mismatch occurs with a character 
in s, we can determine where in the pattern to continue the search for a match 


without moving backwards in s. To formalize this, we define a failure function 
for a pattern, 


Definition: If p = pop --* py- is a pattern, then its failure function, f, is 
defined as 


fU)= largest k <j such that po «+ py = pj-x ** Pj if such ak 2 Oexists 4 
i otherwise. 


For the example pattern above, pat = abcabcacab, we have 


i 8 1 2° 3 4 5 6 7 8 9 
pa a& bh lhe all alk 
fo av -) + 0 1 2 39 -f O 1 


___ From the definition of the failure function, we arrive at the following mule 
for pattern matching: /f a partial match is found such that 
Sy 7 Sis = PO Pj and s;#p; then matching may be resumed by 
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comparing s, and pyy~1)41 if j #0. If j = 0, then we may continue by comparing 
5;41 and po. This pattern-matching rule translates to function FastFind (Program 
2.16). FastFind uses an array of ints, f, to represent the failure function. f is a 
private data member of String. 


L int String:: FastFind(String pat) 

2 {// Determine if pat is a substring of s. 

3 int posP =0, posS=0; 

4 int lengthP = pat.Length(), lengthS = LengthQ); 

5 while ((posP < lengthP) && (posS < lengthS)) 

6 i€ (pat.str{posP] = str(posS)) { // character match 
7 

8 

9 


posP++ ; posS++ 5 
else 
10 if (posP = 0) 
il posS++; 


12 else posP = pat. f[posP -1} +15 
13 if (osP <lengthP) return -1; 
14 else return posS —lengthP; 


Program 2.16: Pattern-matching with a failure function 


Analysis of FastFind: The correctness of FastFind follows from the definition 
of the failure function. To determine the computing time, we observe that lines 7 
and 11 can be executed for a total of at most lengthS times, since in each itera- 
tion posS is incremented by 1 but posS is never decremented in the algorithm. 
As a result, posP can move right on pat at most lengthS times (line 7). Since 
each execution of line 12 moves posP left on pat, it follows that this clause can 
be executed at most /engthS times, since otherwise posP becomes less than 0. 
Consequently, the maximum number of iterations of the while loop is lengthS, 
and the computing time of FastFind is O(lengthS). O 


From the analysis of FastFind, it foltows that if we can compute the failure 
function in O(/engthP) time, then the entire pattern-matching process will have a 
computing time proportional to the sum of the lengths of the string and pattern. 
Fortunately, there is a fast way to compute the failure function. This is based 
upon the following restatement of the failure function: 


-1 ifj=0 
fU) = | F"G - 1) + 1 where m is the least integer k for which pyqj_1)41 = P, 
-{ if there is no & satisfying the above 
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(Note that f(j) = f (j) and f"U) = FF"). This directly yields the function 
of Program 2.17 to compute f. 


1 void String::FailureFunction() A 
2 {Uf Compute the failure function for the pattern *this. 
3. int lengthP = Length() 5 


4 fl0}=-t; ; 
5. for (int j= 15 j <lengthP ; j++) // compute f Lj] 
6 { 


7 inti=ffj-ls 

8 — while ((#(str + j) = 4(str+i+1)) && (i >= O) i =f [15 
9 if (A(sir+ j) == (str +i+ 1) 

10 fUl=iels 

M1 else ff) =-15 


Program 2.17: Computing the failure function 


Analysis of FailureFunction: In each iteration of the while loop the value of i 
decreases (by the definition of f). The variable i is reset at the beginning of each 
iteration of the for loop. However, it is either reset to -1 (when j = 1 or when 
the previous iteration went through line 11), or it is reset to a value | greater than 
its terminal value on the previous iteration (i.e., when the previous iteration went 
through line 10). Since only lengthP - | executions of line 7 are made, the value 
of i therefore has a total increment of at most lengthP ~ 1. Hence it cannot be 
decremented more than fengthP ~ 1} times. Consequently, the while loop is 


iterated at most lengthP — 1 times over the whole algorithm, and the computing 
lime of FailureFunction is O(lengthP). O 


Note that when the failure function is not known in advance, pattem 
matching can be carried out in time O(lengthP + lengthS) by first computing 
Failure Function and then performing a pattern match using function FastFind. 
EXERCISES 


1. Write a function String::Frequency that determines the frequency of 


occurrence of each of the distinct characters in the string. Test your func- 
Won using suitable data. 
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2. Write a function, String::Delete, that accepts two integers, start and length. 
The function computes a new string that is equivalent to the original string, 
except that fength characters beginning at start have been removed. 

3. Write a function, String::CharDelete, that accepts a character c. The func- 
tion retums the string with all occurrences of c removed, 

4. Write a function to make an in-place replacement of a substring w of a 
string by the string x. Note that w may not be of the same size as x. What 
is the complexity of your function? 

5. If x = (x9, 1 Xm—1) and y = (Yo, --s Yy-1) ae strings, where x; and y, are 
letters of the alphabet, then x is less than y if x; = y; forO <i < jand x, <y; 
or if x; = y; for O0Si<m and m<n. Write an algorithm that takes two 
strings x,y and returns either - 1,0, or + } ifx <y, x =y, or x > y respec- 
lively, 

6. (a) Find a string and a pattern for which function Find (Program 2.15) 

takes time proportional to lengthP - lengthS. 

(b) Do part (a) under the assumption that the function Find has been 
modified to check for a match with the last character of the pattern 
first (see text for an explanation of this heuristic). 

7. Compute the failure function for each of the following patterns: 

(a) aaaab 

(b) ababaa 

(c) abaabaabb 


8. Let POP 4,--- Put be a pattern of length n. Let f be its failure function. 
Define f'G) =f) and fr) = ff"), OSJ <n and m > 1. Show, 
using the definition of f, that 


-1 ifj=0 
fQ@Ms [ro- 1)+1 where m is the least integer k for which ppg-1y41 = Pj 
1 if there is no k satisfying the above 


9. The definition of the failure function may be strengthened to 


Fue [largest <j such that py *** pj = Pj-i °° Pj AND P41 FP; 41 
-l if there is noi 20 zallshying @ above 


(a) Obtain the new failure function for the pattern par of the text. 
(b) Show that if this definition for f is used, then algorithm FastFind 
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(Program 2.16) still works correctly. 
(c) Modify algorithm FailureFunction (Program 2.17) to compute 
under this definition. Show that the computing time is stil! O(n), f 
(d) Ase there any patterns for which the observed computing time of 
FastFind is more with the new definition of f than with the old one? 
Are there any for which it is less? Give examples. ‘i 


2.7 REFERENCES AND SELECTED READINGS 


The Knuth, Morris, Pratt patiern-matching algorithm can be found in ‘Fast pat. 
tern matching in strings,” SIAM Journal on Computing, 6:2, 1977, pp. 323-350, 
A discussion of the Knuth Morris Pratt algorithm, along with other string match- 
ing algorithms, may be found in Introduction to Algorithms Second Edition, by 
‘TT. Cormen, C. Leiserson, R. Rivest and C. Stein, McGraw Hill, New York, 2002, 


2.8 ADDITIONAL EXERCISES 


1, 


Write a C++ function to make an in-place reversal of the order of elements 
in the array list. That is, the function should transform fist such that fol- 
lowing the transformation, list[i] contains the element originally in 
list {n - i — 1)}th, where n is the total number of elements in fist. The only 
additional space available to your function is that for simple variables. 
How much time does your function take to accomplish the reversal? 


An m Xn matrix is said to have a saddle point if some entry a [i}[/] is the 
smallest value in row i and the largest value in column j. Write a C++ 


function that determines the location of a saddle point if one exists. What 
is the computing time of your method? 


When all the elements either above or below the main diagonal of a square 
matrix are zero, then the matrix is said to be triangular. Figure 2.8 shows a 
lower and an upper triangular matrix. In a lower triangular matrix, a, with 


n rows, the maximum number of nonzero terms in row i is i+1. Hence, the 
total number of nonzero terms is 72} (i + 1) = n(n + 1)/2. For large nit 
would be worthwhile to save the space taken by the zero entries in the 
upper triangle. Obtain an addressing formula for elements aj; in the lower 
triangle if this lower triangle is stored by rows in an array 4 [a@ + D/2 
with alOll0) being stored in b[0}. What is the relationship between é and j 
for elements in the zero part of A? 
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x MXXXXKXXXK 
Kx x x 
x Xx x x 
x x x non- xX 
x xX zero x zero Xi 
xX non- x x Xx 
x zero Xx zero x Xi 
x x x x 
x x xXx 
IXXXXXXXXXX x 
lower triangular upper triangular 


Figure 2.8: Lower and upper triangular matrices 


4. Leta and be two lower triangular matrices, each with n rows. The total 
number of elements in the lower triangles is n{n + 1). Devise a scheme to 
represent both the triangles in an array c[][n +1}. [Hint: Represent the 
triangle of a as the lower triangle of c and the transpose of b as the upper 
triangle of c.] Write algorithms to determine the values of a[i}|j] and 
b[i}U], OS, j <n, from the array c. 

5. Another kind of sparse matrix that arises often in practice is the tridiagonal 
matrix. In this square matrix, all elements other than those on the major 
diagonal and on the diagonals immediately above and below this one are 
zero (Figure 2.9). 


Figure 2.9: Tridiagonal matrix 


If the elements in the band formed by these three diagonals are represented 
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by rows in an array, b, with a [0){0] being stored in 50], obtain an 
rithm to determine the value of afi][j], 0S i j <n from the array 5, 
A square band matrix @,,q is ann Xn matrix in which all the nonzero terms 
jie in a band centered around the main diagonal. The band includes a~} 
diagonals below and above the main diagonal and also the main diagonal, 


algo. 


(Figure 2.10). 


adiagonals 


upper band 


ncolumns 
main diagonal 
Ana 


Figure 2.10; Square band matrix 


(a) How many elements are there in the band of a,,,? 

(b) What is the relationship between i and j for elements aj; in the band 

Of dy? 

{c) Assume that the band of a,,, is stored sequentially in an array by 

diagonals starting with the lowermost diagonal. Thus, a43 above 

would have the following representation: 

DIO} bit] 612) BL] bL4] b{S] IG} b{7} {8} bI9) bfL0) bE1t) b{12} bf13) 
ae a See a ee ee Se Se 9 $4 

3 x 410 425 G32 G09 41) G22 433 Age ay2 a2, AQ 93 


Obtain an addressing formula for the location of an element aj; in 
the lower band of @,,4, €.g., LOC(a) = 0, LOC (a) = 1 in the 
example above. 
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7. A generalized band matrix a, is ann Xn matrix a in which all the 
nonzero terms lie in a band made up of a — 1 diagonals below the main 
diagonal, the main diagonal, and b — 1 diagonals above the main diagonal 
(Figure 2.11). 

(a) How many elements are there in the band of ay,44? 


(b) What is the relationship between i and j for elements a), in the band 
Of anab? 

(c) Obtain a sequential representation of the band of a,,4 in the one- 
dimensional array c. For this representation, write a C++ function 
value (n,a,b,i,j,c) that determines the value of element aj; in the 
matrix p4,4- The band of aya is represented in the array c. 


main diagonal 


Anab 


Figure 2.11: Generalized band matrix 


8. [Programming Project] There are a number of problems, known collec- 
tively as ‘‘random walk’ problems, that have been of longstanding interest 
to the mathematical community. All but the most simple of these are 
extremely difficult to solve and for the most part they remain largely 
unsolved. One such problem may be stated as 


A (drunken) cockroach is placed on a given square in the middle of a 
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tile floor in a rectangular room of size n xm tiles. The bug wanders 
(possibly in search of an aspirin) randomly from tile to tile throughout 
the room. Assuming that he may move from his present tile to any of 
the eight tiles surrounding him (unless he is against a wall) with equal 


probability, how long will it take him to touch every tile on the floor 
at least once? 


Hard as this problem may be to solve by probability theory techniques, it is 
easy to solve using the computer. The technique for doing so is called 
“simulation.” This technique is widely used in industry to predict traffic 
flow, inventory control, and so forth. The problem may be simulated using 
the following method: 


Ann Xm array count is used to represent the number of times our cock. 
roach has reached each tile on the floor. All the cells of this array are ini- 
tialized to zero. The position of the bug on the floor is represented by the 
coordinates (ibug,jbug). The eight possible moves of the bug are 
represented by the tiles located at (ibug + imove [k], jbug + jmove{k)), 


where 0<k <7 and 
imove(O}=  -1 Jjmove[0] = 1 
imove[1J= 0 jmove{1) = 1 
imove(2} = 1 jmove(2) = 1 
imove[3} = 1 jmove(3] = 0 
imove[4J= 1 jmove[4J=  -1 
imove[5] = 0 jmove[5}= 1 
imove[6J= — -1 jmove[6)= — ~1 
imove[T]= 1 jmove(T}= 0 


A random walk to one of the eight given squares is simulated by generat- 
ing a random value for k lying between 0 and 7. Of course, the bug cannot 
move outside the room, so coordinates that lead up a wall must be ignored 
and a new random combination formed. Each time a square is entered, the 
count for that square is incremented so that a nonzero entry shows the 
number of times the bug has landed on that square so far. When every 
square has been entered at least once, the experiment is complete. 


Write a program to perform the specified simulation experiment. Your pro- 
gram MUST: 


(a) handle all values of n and m, 2<n<40,2<m<20 


(b) perform the experiment for (1) n = 15, m = 15, starting point: (9,9) 
and (2) n = 39, m = 19, starting point: (0,0) 
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(c) have an iteration Jimit, that is, a maximum number of squares the 
bug may enter during the experiment (this avoids getting hung in an 
infinite loop); a maximum of 50,000 is appropriate for this exercise 

(d) for each experiment, print (1) the total number of legal moves that 
the cockroach makes and (2) the final count array (this will show the 
density of the walk, that is, the number of times each tile on the floor 
was touched during the experiment). 

(Have an aspirin.) This exercise was contributed by Steve Olson. 
[Programming Project| Chess provides the setting for many fascinating 
diversions that are quite independent of the game itself. Many of these are 
based on the strange L-shaped move of the knight. A classical example is 
the problem of the knight's tour, which has captured the attention of 
mathematicians and puzzle enthusiasts since the beginning of the 
eighteenth century. Briefly stated, the problem is to move the knight, 
beginning from any given square on the chessboard, in such a manner that 
it travels successively to all 64 squares, touching each square once and 
only once. It is convenient to represent a solution by placing the numbers 
1,2, +++, 64 in the squares of the chessboard indicating the order in which 
the squares are reached. Note that it is not required that the knight be able 
to reach the initial position by one more move; if this is possible, the 
knight’s tour is called re-entrant. One of the more ingenious methods for 
solving the problem of the knight’s tour was that given by J. C. Warnsdorff 
in 1823. His rule was that the knight must always be moved to one of the 
squares from which there are the fewest exits to squares not already 
traversed. Use Warnsdorff’s rule to construct a particular solution to the 
problem by hand before reading any further. 


The most important decisions to be made in solving a problem of this type 
are those concerning how the data is to be represented in the computer. 
Perhaps the most natural way to represent the chessboard is by an 8 x 8 
array board, as shown in Figure 2.12. The eight possible moves of a 
knight on square (4,2) are also shown in this figure. In general a knight at 
(i,j) may move to one of the squares (6-2, +1), (i-1,j +2) 
G+tljy+2), G+27+, G@+2f/-D, G@+hj-2, G-1j/-2), 
(i-2,) - 1). Notice, however, that if (i,j) is located near one of the edges 
of the board, some of these possibilities could move the knight off the 
board, and, of course, this is not permitted. The eight possible knight 
moves may conveniently be represented by two arrays kimov! and ktmov2: 
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Figure 2,12: Legal moves for a knight 


ktmovI — ktmov2 


-1 2 
1 2 
2 1 
2 -i 
1 2 

-1 -2 

-2 +) 


Then a knight at (i,j) may move to (i + ktmovi [k],j + ktmov2[k]), where k 


is some value between 0 and 7, provided that the new square lies on the 
chessboard. 


An algorithm to solve the knight's tour problem using Warnsdoff’s tule is 
described below, 


(a) [Initialize chessboard] For 0 < i,j $7, set board {i} to 0. 
(b) [Set starting position] Read and print ij and then set board[i){i] 0 
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1, 

(c) {Loop] For 2 <m < 64, do steps (d) through (g). 

(d) [Form set of possible next squares} Test each of the eight squares 
one knight’s move away from (i,j) and form a list of the possibilities 
for the next square (nexti [/],nextj{/}). Let npos be the number of 
possibilities. (That is, after performing this step we will have 
nexti(1] = i+konovl(k) and nextj[/] = j+ktmov2[k], for certain 
values of k between 0 and 7. Some of the squares ( + ktmov/[k], 
j + ktmov2[k]) may be impossible for the next move either because 
they lie off the chessboard or because they have been previously 
occupied by the knight (i.e., they contain a nonzero number). In 
every case we will have 0 Snpos <8.) 


(e) [Test special cases} If npos = 0, the knight’s tour has come to a 
premature end; report failure and then go to step (h). If npus = 1, 
there is only one possibility for the next move; set min = 0 and go 
right to step (g). 


(f) [Find next square with minimum number of exits\ For 1 <1 <npos 
set exits{/] to the number of exits from square (nexti [! ],nextj(!)). 
That is, for each of the values of /, examine each of the next squares 
(nexti [1]+ktmovl[k], nextj [1] + ktmov2[k}) to see if it is an exit 
from (nexti{/],nextj[i]), and count the number of such exits in 
exits (1). (Recall that a square is an exit if it lies on the chessboard 
and has not been previously occupied by the knight.) Finally, set 
min to the location of the minimum value of exits. (There may be 
more than one occurrence of the minimum value of exits. If this 
happens, it is convenient to let min denote the first such occurrence, 
although it is important to realize that by so doing we are not actu- 
ally guaranteed finding a solution. Nevertheless, the chances of 
finding a complete knight’s tour in this way are remarkably good, 
and that is sufficient for the purposes of this exercise.) 

(g) (Move knight] Set i = nexti [min], j = nextj[min], and board{i)lj] 
= m. Thus, (i,j) denotes the new position of the knight. and 
board{i }[j ] records the move in proper sequence. 

(h) (Output) Output board showing the solution to the knight’s tour and 
then terminate the algorithm. 


The problem is to write a C++ program that corresponds to this algorithm. 
This exercise was contributed by Legenhausen and Rebman. 


CHAPTER 3 


Stacks and Queues 


3.1 Templates in C++ 


In this section, we introduce the concept of templates. This is a mechanism pro- 
vided by C++ to make classes and functions more reusable. We shall discuss 
both template functions and template classes. 


3.1.1 Template Functions 


Consider function SelectionSort of Program 1.6, which sorts an array of integers 
using the selection sort method. Suppose, we now wish to use selection sort to 
sort an array of floating point numbers. The code of Program 1.6 can be used to 
do this if we change int *a to float *a in the function header. From a practical 
standpoint, this can be achieved by using a text editor to replicate the code for 
function SelectionSort and then making the stated change. This process would 
have to be repeated if we next wished to sort an array of, say, characters. 
Considerable software development time and money can be saved if we 
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use templates or parameterized types. A template may be viewed as a variable 
that can be instantiated to any data type, irrespective of whether this data type is 
a fundamental C++ type or a user-defined type. The template function Selection- 
Sort (Program 3.1) is defined using the parameterized type T. 


1 template <class T> 
2 void SelectionSort (T *a, const int n) 
3 {(# Sort a[0) to a[n-1] into nondecreasing order. 


4 for (int i=O;i<n3it+) 

5 { 

6 intj=i; 

7 # find smallest integer in a[i] to a{n — 1) 
8 for (intk=i+1;k<n;k++) 

9 if (afk] < ali) j=ks 
10 swap (ali), aliDs 


Program 3.1: Selection sort using templates 


Function SelectionSort can now be used quite easily to sort an array of ints or 
floats as shown in Program 3.2. Function SelectionSort is instantiated to the type 
of the array argument that is supplied to it. For example, the first call to Selec- 
tionSort knows that it is to sort an array of floating point numbers because array 
farray is an array of floating point numbers. 


float farray[100); 
int intarray[250]; 


/f assume that the arrays are initialized at this point 
SelectionSort(farray, 100); 
SelectionSort(intarray, 250); 


Program 3.2: Code fragment illustrating template instantiation 


Observe that SelectionSort uses the operator "<" to compare two objects of type 
T in line 9. So long as this operator (as well as others used by swap) is defined 
for the data type T, Program 3.1 may be used to sort an array of type T. 
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Suppose we want to use Program 3.1 to sort an array of Rectangles in non. 
decreasing order of their areas. We cannot instantiate T to Rectangle as we did 
with int and float in Program 3.2 because operator “<" of line 9 is undefined for 
Rectangle. This can be remedied by overloading operator< so that it Tetums 
true if the first operand has Jess area than the second and false, otherwise, Of 
course, additional operators used by swap also must be defined for Rectangle 
unless the C++ default (if any) for these operators will suffice. 

A template function that we shall find quite useful in this text is one to 
change the size of a !-dimensional array. We have already seen two applications 
for array resizing (polynomial and matrix addition (Programs 2.6 and 2.13)). Pro- 
gram 3.3 gives a template function that changes the size of a 1-dimensional array 
of type T from oldSize to newSize. 


template <class T> 
void ChangeSize! D(T*& a, const int oldSize, const int newSize) 


if (newSize < 0) throw "New length must be >= 0"; 


T* temp = new T [newSize };, 

int number = min(oldSize, newSize); 
copy(a, a + number, temp); 

delete (] a; 

a= temp; 


i new array 
# number to copy 


Hf deallocate old memory 


} 


Program 3.3: Template function to change the size of a 1-dimensional array 


3.1.2 Using Templates to Represent Container Classes 


A container class is a class that represents a data structure that contains or stores 
a number of data objects. Objects can usually be added to or deleted from a con- 
tainer class. The array is an example of a container class as it is used to store 4 
number of objects. We begin by introducing the container class Bag. Our 
specification for the class Bag is that it is a data structure into which objects can 
be inserted and from which objects can be deleted, like a bag of groceries. A bag 
can have multiple occurrences of the same element, but we do not care about the 
position of an element; nor do we care which element is removed when a delete 
operation is performed. We implement Bag by using a C++ array to hold its 
objects. Program 3.4 contains the class definition of a Bag of integers. The 
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default initial capacity of a bag is 10. When a Bag function is unable to perform 
its task, it is to throw an exception. 


class Bag 
public: 
Bag (int bagCapacity = 10); // constructor 
“Bag; # destructor 
int Size() const; #/ return number of elements in bag 
bool /sEmpry{) const; // return true if the bag is empty; false otherwise 
int Element() const; // return an element that is in the bag 
void Push(const int); H insert an integer into the bag 
void Pop(); H delete an integer from the bag 
private: 
int *array; 
int capacity; H capacity of array 
int top; array position of top element 
43 


Program 3.4: Definition of the class Bag containing integers 


We implement insertion into the Bag by storing the element that is to be 
inserted into the first available position in the array. In case the array is full, its 
capacity is doubled. Deletion is implemented by deleting the element in the 
middle position of the array, and then moving all elements to its right, one posi- 
tion to the left. The function Element is implemented so as to return the clement 
that will be deleted in case a Pop is done. Our implementation decisions for 
choosing the positions at which an element will be inserted or deleted as well as 
for choosing the element returned by Element are arbitrary. We could have just 
as easily decreed that elements should be inserted into the middle position, 
deleted from the last position, and that Element return the integer in the first posi- 
tion of the array. Program 3.5 contains our implementation of Bag operations. 

Most of the operations of Bag need no explanation. We have declared 
member functions Size(), /sEmpty(), and Element as inline because of their small 
length (each contains only two lines of code). This eliminates the overhead of 
performing a function call. The three functions have the const attribute as they 
do not change the bag object upon which they operate. 
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Bag::Bag (int bagCapacity): capacity ( bagCapacity ) { 
if (capacity < 1) throw “Capacity must be > 0"; 
array = new int[capacity]; 
top =-13 


} 

Bag:+ Bag() { delete [ ] array;} 

inline int Bag::Size() const (return top+};} 

inline bool Bag::/sEmpty() const {return size == 03} 


inline int Bag::Element() const { 
if (IsEmpty ()) throw "Bag is empty"; 
return array [0}; 


} 


void Bag::Push(const int x) { 


if (capacity = = top+1) ChangeSize!D(array, capacity, 2 * capacity); 
capacity *= 2; 
array[++top] = x; 


) 


void Bag::Pop() { 


if (/sEmpty()) throw "Bag is empty, cannot delete"; 
int deletePos = top / 2; 


copy (array + deletePos + \, array + top +1, array + deletePos); 
# compact array 
fop--5 


} 


Program 3.5: Implementation of operations of Bag 


As we have defined it, Bag can be used to store integers only. We would 
like to implement Bag using templates so that Bag can be used to store objects of 
any data type. Container classes are particularly suitable for implementation 
using templates because the algorithms for basic container class operations are 
usually independent of the type of objects that the container class contains. Fo! 
example, the algorithm to add an element to Bag does not depend on the tyPé of 
its elements. Program 3.6 contains a template class definition for Bag 


133 Templates 


Program 3.7 contains the implementation of some of its operations. These are 
obtained from Programs 3.4 and 3.5, respectively, by prefixing the class 
definition of Bag and the definitions of all its member functions that are outside 
the class (in our example, all member functions are defined outside the class 
definition) with the statement: 


template<class 7> 


Next, int is replaced by 7 when it refers to an object being stored in Bag. 
Finally, in all member function headers, Bag:: is replaced by Bag<T>::. Notice, 
also, that the single input parameter to Push is made a constant reference. This is 
done because it is now possible that T represents a large object, and considerable 
overhead may be incurred if it is passed by value. 


template <class 7> 
class Bag 


public: 
Bag (int bagCapacity = 10); 
“Bag0s 


int Size() const; 
bool /sEmpty() const; 
T& Element() const; 


void Push(const T&); 
void Pop(); 


private: 
T *array; 
int capacity; 
int rop; 


8 


Program 3.6: Definition of template class Bag 


The following statements instantiate the template class Bag to int and Rectangle. 
respectively. So, a is a Bag of integers and r is a Bag of Rectangles. 

Bag<int> a; 

Bag<Rectangle> rv; 
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template <class T> 


Bag<T>::Bag (int bagCapacity): capacity { bagCapacity ) { 
if (capacity < 1) throw "Capacity must be > 0"; 


new T [capacity]; 
13 
} 
template <class T> 


Bag<T>:; Bag() {delete { ) array; } 


template <class T> 
void Bag<T>::Push(const T& x) { 
if (capacity = = top+1) 


ChangeSize!D(array, capacity, 2 * capacity); 
capacity * = 2; 


array[++top) = x; 


template <class T> 
void Bag<T>::Pop() { 
if (sEmpry()) throw "Bag is empty, cannot delete"; 
int deletePos = top | 2; 
copy (array + deletePos + 1, array + top +1, array + deletePos); 
# compact array 


array (top—).“T();_ Hf destructor for T 
} 


Program 3.7: Implementation of some operations of Bag 
3.2. THE STACK ABSTRACT DATA TYPE 


We shall now study two data structures that are frequently used in computer pe 
grams. These data structures, the stack and the queue, are special cases of the 
more general data structure ordered list that we discussed in Section 2.3. In this 
section we begin by detining the ADT Stack and follow with its implementation. 
In Section 3.3 we look at the queue. Pet 

A stack is an ordered list in which insertions (also known as agaitions 
Puts, and pushes) and deletions (also known as removals and pops) are made # 
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one end called the top. Given a stack S = (ag, *~* . @n-). we Say that do is the 
bottom element, a,_; is the top element, and a; is on top of element 
a;.;,0 <i <n. The restrictions on the stack imply that if we add the elements A, 
B, C, D, E to the stack, in that order, then E is the first clement we delete from 
the stack. Figure 3.1 illustrates this sequence of operations. Since the last ele- 
ment inserted into a stack is the first element removed, a stack is also known as a 
Last-In-First-Out (LIFO) list. 


E top 
D top |D D top 
Che-top |C Cc Cc 
Ble-top |B B B B 
A top [A A A A A 
add add add add pop 


Figure 3.1: Inserting and deleting elements in a stack 


Example 3.1 [System stack]: Before we discuss the stack ADT, we look at a spe- 
cial stack, called the system stack, that is used by a program at runtime to pro- 
cess function calls. Whenever a function is invoked, the program creates a struc- 
ture, referred to as an activation record or a stack frame, and places it on top of 
the system stack. Initially, the activation record for the invoked function con- 
tains only a pointer to the previous stack frame and a return address. The previ- 
ous stack frame pointer points to the stack frame of the invoking function; the 
returm address contains the location of the statement to be executed after the 
function terminates. Since only one function executes at any given time, the 
function whose stack frame is on top of the system stack is chosen. If this func- 
tion invokes another function, the local variables and the parameters of the 
invoking function are added to its stack frame. A new stack frame is then 
created for the invoked function and placed on top of the system stack. When 
this function terminates, its stack frame is removed and the processing of the 
invoking function, which is again on top of the stack, continues. A simple exam- 
ple illustrates this process. 

Assume that we have a main function that invokes function al. Figure 
3.2(a) shows the system stack before al is invoked; Figure 3.2(b) shows the sys- 
tem stack after al has been invoked. Frame pointer fp is a pointer to the current 
stack frame. The system also maintains separately a stack pointer, sp, which we 
have not illustrated. 
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| PSE re | 
r—| previous frame pointer fp 


return address al 


local variables 


—} previous frame pointer 


mal return address ‘main 
(b) 


Figure 3.2: System stack after function call 


Since all functions are stored similarly in the system stack, it makes no 
difference if the invoking function calls itself. That is, a recursive call requires 
no special strategy; the runtime program simply creates a new stack frame for 
each recursive call. However, recursion can consume a significant portion of the 


memory allocated to the system stack; it could consume the entire available 
memory. 0 


Our discussion of the system stack suggests a basic set of operations, 
including insert an item, delete an item, and check for stack empty. ‘These are 
given in the ADT specification (ADT 3.1). When an ADT function is unable to 
pesform its task, it is to throw an exception. 

The easiest way to implement the stack ADT is by using an array, Say 
stack{]. The first, or bottom, element of the stack is stored in stack {0}, the 
second in stack {1}, and the ith in stack [i-1]. Associated with the array is a a 
able, top, which points to the top element in the stack. Initially, top is set 10 ~’ 


tu denote an empty stack. This results in the following data member declarations 
and constructor definition of Stack: 


private: 
T stack; # array for stack elements 
int 1p; / array position of top element 
int capacity; 


di capacity of stack array 
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template <class T> 
class Stack 
{// A finite ordered list with zero or more elements. ; 
public: 
Stack (int stackCapacity = 10); 
4 Create an empty stack whose initial capacity is stackCapacity. 


bool /sEmpty() const; 
#4 Yf number of elements in the stack is 0, return true else return false. 


T& Top() const; 
#/ Return top element of stack. 


yoid Push (const 7& item); 
// Insert item into the top of the stack. 


void Pop(); 
4 Delete the top element of the stack. 
b 


ADT 3.1: Abstract data type Stack 


template <class T> 
Stack<T>::Stack (int stackCapacity) : capacity (stackCapacity) 
{ 


if (capacity < 1) throw "Stack capacity must be > 0"; 
stack = new T (capacity); 
top =-1; 
The member functions /sEmpty( ) and Top( ) are implemented as follows: 


template <class T> 
inline bool Stack<T>::/sEmpty() const { return fop == —1;} 


template <class T> 
inline T& Stack<T>::Top() const 


if (/sEmprty ()) throw "Stack is empty"; 
return stack [top |; 
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The Push and Pop functions are shown in Programs 3.8 and 3.9. 


template <class T> 


void Stack<T>::Push (const T& x) 
{# Add x to the stack. 


if (top = = capacity - 1) 


ChangeSizel D(stack, capacity, 2*capacity); 
capacity *= 23 


stack{++top) = x; 


Program 3.8: Adding to a stack 


template <class T> 
void Stack<T>::Pop() 


{// Delete top element from the stack. 


if (/sEmpry()) throw "Stack is empty. Cannot delete."; 
stack top——).T();_ #f destructor for T 


Program 3.9: Deleting from a stack 


EXERCISES 


1B 


nN 


Extend the stack ADT by adding functions to output a stack; split a stack 
into two stacks with one containing the bottom half elements and the 
second the remaining elements; and to combine two stacks into one by 
placing all elements of the second stack on top of those of the first stack. 
Write C++ code for your new functions. 

Consider the railroad switching network given in Figure 3.3. Railroad cars 
numbered [, 2, 3... ., are initially in the top right track segment (in this 
order, left to right). Railroad cars can be moved into the vertical track seg 
ment one at a time from either of the horizontal segments and then moved 
from the vertical segment to any one of the horizontal segments. The vert: 
cal segment operates as a stack as new cars enter at the top and cars depart 
the vertical segment from the top. For instance, if n = 3, we could move 
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car | into the vertical segment, move 2 in, move 3 in, and then take the cars 
out producing the new order 3, 2, 1. For 2 = 3 and 4 what are the possible 
permutations of the cars that can be obtained? Are any permutations not 
possible? 


Figure 3.3: Railroad switching network 


3.3. THE QUEUE ABSTRACT DATA TYPE 


A queue is an ordered list in which insertions (also called additions, puts, and 
pushes) and deletions (also called removals and pops) take place at different 
ends. The end at which new elements are added is called the rear, and that from 
which old elements are deleted is called the front. The restrictions on a queue 
imply that if we insert A, B, C, D, and E in that order, then A is the first element 
deleted from the queue. Figure 3.4 illustrates this sequence of events. Since the 
first element inserted into a queue is the first element removed, queues are also 
known as First-In-First-Out (FIFO) lists. The ADT specification of the queue 
appears in ADT 3.2. When an ADT function is unable to perform its task, it is to 
throw an exception. 

Analogous to the representation of a stack in Section 3.2, we may use an 
array queue (] with the first (or front) element of the queue in qiewe [0], the next 
in queue {1], and so on. Figure 3.5(a) shows a 3-element queue repesented in this 
way and Figure 3.5(b) shows the queue after an element has been deleted. Figure 
3.5(c) shows the result of adding an element to the queue of Figure 3.5(b). Since 
a delete or pop removes the front element, which is in guese [0], each delete 
reqiures us to shift the remaining elements to the left. Hence, it takes O(n) time 
to pop an element from a queue that has n elements; an insertion or push, on the 
other hand, can be done in ©(1) time exclusive of the time for any array resizing 
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Figure 3.4: Inserting and deleting elements in a queue 


template <class T> 
class Queue 
{// A finite ordered list with zero or more elements. 
public: 
Queue(int gueueCapacity = 10); 
1 Create an empty queue whose initial capacity is queueCapacity 
bool /sEmprty() const; 
J Vf number of elements in the queue is 0, return true else return false. 
T& Front() const; 
/# Return the element at the front of the queue. 
T& Rear() const; 
# Return the element at the rear of the queue. 
void Push (const T& item); 
4 Insert item at the rear of the queue. 
void Pop(); 


4 Delete the front element of the queue. 
k 


ADT 3.2: Abstract data type Queue 


that may be needed. 

To pop an element in Q(1) time, we must relax the requirement that 
quene {Q] contain the front element of the queue. With this relaxation, we use & 
variable, front, 10 keep track of the location of the front element, The queue 
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Figure 3.5: Queues represented with front element in queue [0} 


elements are in queue [front], -- +, queue [rear]. Figure 3.6 shows three queues 
represented in this way. The queue of Figure 3.6(b) results from the deletion of 
the front element of the queue of Figure 3.6(a) and that of Figure 3.6(c) results 
from adding an element to the queue of Figure 3.6(b). 


rear rear rear 


front a front 
Ajaic] - Ble]! — 
(a) (b) (c) 


Figure 3.6: Queues represented with front element in queue [front] 


oa 
Hi 
, 


Suppose that the capacity of the array queue is capacity. The representa- 
tion of Figure 3.6 runs into a problem when rear equals capacity — | and front > 
Q as in Figure 3.7(a). How do we add an element to this configuration without 
increasing array capacity? One possibility is to shift all elements to the left end 
of the queue (as in Figure 3.7(b)) and create space at the right end. This shifting 
takes time proportional to the size (i.e., number of elements) of the queue. 

The worst-case add and delete times (assuming no array resizing is needed) 
become ©(1) when we permit the queue to wrap around the end of the array. At 
this time it is convenient to think of the array positions as arranged in a circle 
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front rear front rear 


(a) Before shift (b) After shift 


Figure 3.7: Shifting queue elements to the left 


(Figure 3.8) rather than in a straight line (Figure 3.7). In Figure 3.8, we have 
changed the convention for the variable front. This variable now points one 
position counterclockwise from the location of the front element in the queue. 
‘The convention for rear is unchanged. This change simplifies the codes slightly. 


rear 


? 3 
e 
oi? Say 


Figure 3.8: Circular queue 


‘When the array is viewed as a circle, each array position has a next and & 


Previous position. The position next to position capacity — 1 is 0, and the posi- 
tion that precedes 0 is capaciry Tere 


edes 0 i ‘ity ~ 1. When the queue rear is at capacity — 1, the 
next element is put into position 0. To work with a circular queue, we must be 
able to move the variables front and rear from their current position to the next 
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position (clockwise). This may be done using code such as 


if (rear == capacity ~ 1) rear = 0; 
else rear++; 


Using the modulus operator, which computes remainders, this code is equivalent 
to (rear+1) % capacity. With our conventions for front and rear, we see that the 
front element of the queue is located one position clockwise from front and the 
rear element is at position rear. 

Suppose we implement the ADT Queue using an array in a circular 
fashion. We may use the following data member declarations and constructor. 


private: 
T *queue; 4 array for queue elements 
int front, # one counterclockwise from front 
rear, // array position of rear element 


capacity; // capacity of queue array 


template <class 7> 
Queue<T>::Queue (int queueCapacity) : capacity (queueCapacity) 


if (capacity < 1) throw “Queue capacity must be > 0"; 
queue = new T [capacity]; 
front = rear = 0; 


} 


To determine a suitable test for an empty queue, we experiment with the 
queues of Figure 3.8. To delete an element, we advance front one position clock- 
wise and to add an element, we advance rear one position clockwise and insert 
at the new position. If we perform 3 deletions from the queue of Figure 3.8(c) in 
this fashion, we will see that the queue becomes empty and that front = rear. 
When we do 5 additions to the queue of Figure 3.8(b), the queue becomes full 
and front = rear. So, we cannot distinguish between an empty and a full queue. 
To avoid the resulting confusion, we shall increase the capacity of a queue just 
before it becomes full. Consequently, front == rear iff the queue is empty. The 
member functions /sEmpry( ), Front( ), and Rear( ) are implemented as follows: 


template <class 7> 
inline bool Queue<T>::IsEmpty() {return front == rears} 


template <class T> 
inline T& Queue<T>::Front) 
{ 
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if (isEmpry()) throw “Queue is empty. No front element"; 


return queue [(front + 1) % capacity }; 
} 


template <class T> 
inline T& Queue<T>::Rear() 


if (IsEmpty()) throw “Queue is empty. No rear element"; 
return queue {rear}; 


Program 3.10 gives the code to add an element to a queue. This code uses custom 
code to double the capacity of queue when necessary. To see the need for cus- 
tom code to double capacity, consider the full queue of Figure 3.9(a). This figure 
shows a queue with seven elements in an array whose capacity is 8. To visualize 
array doubling when a circular queue is used, it is better to flatten out the array 


as in Figure 3.9(b). Figure 3.9(c) shows the array after array doubling by 
ChangeSize1D (Program 3.3). 


template <class T> 
void Queue<T>::Push(const& x) 
{/! Add x at rear of queue. 
if ((rear + 1) % capacity == front) 
{/ queue full, double capacity 
; #1 code to double queue capacity comes here 


rear = (rear + 1) % capacity; 
queue [rear | = x3 


} 


Program 3.10: Adding to a queue 


To get a proper circular queue configuration, we must slide the elements in 
the right segment (i.e., elements A and B) to the right end of the array as in Fig- 
ure 3.10(d). The array doubling and the slide to the right together copy at most 
2ecapacity —2 elements. The number of elements copied can be limited to 
capacity ~ 1 by customizing the array doubling code so as to obtain the 
configuration of Figure 3.10(e). This configuration may be obtained as follows: 


(1) Create a new array newQueue of twice the capacity. 
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F queue (0) (1} (2) (3) (4) (5) 16 (71 
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(a) A full circular queue 
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front =5, rear =4 
(c) After array doubling 
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front = 13, rear =4 
(d) After shifting right segment 


(O) (1) (2) (3) (4) (5) (6) (71 (8) (9) (20) (11) [12] (13) [143 (15) 
2 ! a ee 
front = 15, rear =6 
(e) Alternative configuration 


Figure 3.9: Doubling queue capacity 


146 Stacks and Queues 


{2) Copy the second segment (ie., the elements queue [front +1] through 
queue {capacity—1}) to positions in newQuexe beginning at 0. 

(3) Copy the first segment (i.e., the elements queue (0) through queue {rear}) 
10 positions in newQueue beginning at capacity—front—1. 


The code of Program 3.1! obtains the configuration of Figure 3.10(e). Pro- 
gram 3.12 gives the code to delete or pop an element from a queue. Its complex. 
ity is (1). 


Hf allocate an array with twice the capacity 
T* newQueue = new T(2*capacity }; 


Ht copy from queue to newQueue 
int start = (front + 1) % capacity; 
if (start < 2) 

ino wrap around 

copy (queue + start,queue + start + capacity — 1, newQueue); 
else 
{// queue wraps around 

copy (queue + start,queue + capacity, newQueue ); 

copy (queue queue + rear + 1, newQueue + capacity — start); 


A switch to newQueue 
front = 2+*capacity — 1; 
rear = capacity — 2; 
capacity *= 25 

delete [} queue; 

queue = newQueue;, 


Program 3.11: Doubling queue capacity 


One way to use all positions in the array queue is to use an additional vari- 
able. /astOp, (0 record the last operation performed on the queue. This variable 
is initialized to “Pop.” Following each addition, it is set to ‘‘Push’’ and follow- 
ing each deletion, it is set to ‘Pop.’ Now when front == rear, we can determine 
whether the queue is empty or full by examining the value of /astOp. If lastOp is 
“Push.” then the queue is full. Otherwise, it is empty. The use of the variable 
fastOp as described above does, however, slow down the Push and Pop func- 
tions. Since the Push and Pop functions will be used many times in any problem 


The Queue Abstract Data Type 147 


template <class 7> 
void Queue<T>::Pop() 
{// Delete front element from queue. 


if (/sEmpty() throw “Queue is empty. Cannot delete."; 
front = (front + 1) % capacity; 
queue [front].-T();_ // destructor for T 


Program 3.12: Deleting from a queue 


involving queues, the loss of one queve position will be more than made up for 
by the reduction in computing time. Hence, we favor the implementations of 
Programs 3.10 and 3.12 over those that result from the use of the variable Las- 


10Op. 


EXERCISES 


1. 


Rewrite functions Push and Pop (Programs 3.10 and 3.12) using the vari- 
able lastOp as discussed in this section. The queue should now be able to 
hold up to capacity elements. The complexity of each of your functions 
should be (1) (exclusive of the time taken to double queue capacity when 
needed). 


To the class Queue, add functions to return the size and capacity of a 
queue. 


To the class Queue, add a function to split a queue into two queues. The 
first queue is to contain every other element beginning with the first; the 
second queue contains the remaining elements. The relative order of queue 
elements is unchanged. What is the complexity of your function? 


To the class Queue, add a function to merge two queues into one by alter- 
nately taking elements from each queue. The relative order of queue ele- 
ments is unchanged. What is the complexity of your function? 


A double-ended queue (deque) is a linear list in which additions and dele- 
tions may be made at either end. Obtain a data representation mapping a 
deque into a one-dimensional array. Write C++ template functions to add 
and delete elements from either end of the deque and also to return an ele- 
ment from either end. 


A linear list is being maintained circularly in an array with front and rear 
set up as for circular queues. 


(a) Obtain a formula in terms of the array capacity, front, and rear, for 


148 Stacks and Queues 


the number of elements in the list. 
(b) Write a C++ template function to delete the kth element in the list, 


(c) Write a C++ template function to insert an element y immediately 
after the kth element. Use array doubling in case you need to 
increase the capacity of the element array. 


(d) Develop a complete template class List that includes the functions of 
(b) and (c) as well as a constructor and destructor. Your class also 
should include functions to return the ith element in the list and the 


size of the list. An /sEmpty function should be included as well. Test 
all functions. 


What is the time complexity of your functions for (b) and (c)? 


3.4 SUBTYPING AND INHERITANCE IN C++ 


Inheritance is used to express subtype relationships between ADTs. This is also 
referred to as the IS-A relationship. We say that a data object of Type B IS-A 
data object of Type A if B is more specialized than A or A is more general than B; 
that is, all Bs are As, but not all As are Bs; or the set of all objects of Type B is a 
subset of the set of all objects of Type A. For example, Chair IS-A Furniture, 
Lion IS-A Mammal, Rectangle IS-A Polygon. 

Consider the relationship between Bag and Stack. A bag is simply a data 
structure into which elements can be inserted and from which elements can be 
deleted. A stack is also a data structure into which elements can be inserted and 
from which elements can be deleted. However, a stack is more specialized in that 
it requires that elements be deleted in last-in first-out order. So, a stack can be 
used instead of a bag, but a bag cannot be used for an application that requires a 
stack. So, Stack IS-A Bag or Stack is a subtype of Bag. The IS-A relationship is 
a fundamental, conceptual relationship between the specifications of two ADTs. 
The relationship is not affected by the implementation of either ADT. So, Stack 
1S-A Bag is true, even if their implementations are changed (as long as their 
ADT specifications are met). 

C++ provides a mechanism to express the IS-A relationship called public 
inheritance. Program 3.13 shows how public inheritance is implemented in C+. 
The declaration “: public Bag" appended to class name "Stack" implies that 
Stack publicly inherits from Bag. Here, Bag, representing the more general ADT 
is called the base class, while Stack, representing the more specialized ADT is 
culled the derived class. You will notice that some member functions are pre- 
ceded by the keyword virtual. We will discuss this in the next chapter. 


An important consequence of inheritance is that the derived class (Stack) inherits 
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class Bag 


public: 
Bag (int bagCapacity = 10); 
virtual ~Bag(); 


virtual int SizeQ) const; 
virtual bool /sEmpty() const; 
virtual int Element() const; 


virtual void Push(const int); 
virtual void Pop(); 


protected: 
int *array; 
int capacity; 
int op; 
oF 
class Stack : public Bag 
{ 
public: 
Stack (int stackCapacity = 10); 
“Stack(); 
int Top() const; 
void Pop(); 
% 


Program 3.13: Definition of Bag and Stack 


all the members (data and functions) of the base class (Bag). However, only the 
non-private members of the base class are accessible to the derived class. This 
means that any members in Bag that are protected or public may be accessed by 
Stack. So, array, capacity, and top may be accessed by Stack even though they 
are not defined in the class definition of Stack. Similarly, Size, IsEmpty, Element, 
and Push are accessible to Stack even though they are not defined in Stack. 

Another important consequence of public inheritance is that inherited pub- 
lic and protected members of the base class have the same level of access in the 
derived class as they did in the base class. So all three data members are pro- 
tected members of Stack. The functions Size, IsEmpty, Element, and Push are all 
public members of Stack. 
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The member functions inherited by Stack from Bag have the same Proto. 
types. This is a reuse of the interface of a base class. Notice also that the only 
operation that has a different implementation in Bag and Stack is Pop. The other 
operations all have identical implementations. C++ allows us to reuse the base. 
class implementation of an operation in a derived class. In the event that the 
implementation of a derived class operation must be different from the base class 
implementation, it is also possible to override the base class implementation, In 
our example, only Pop is redefined in the derived class. (The constructor and 
destructor cannot be inherited and therefore must also be redefined in the derived 
class.) The implementations of all Bag operations are assumed to be identical to 
those in Program 3.5. In Program 3.14, we show the member functions of Stack 
that need to be reimplemented. The implementations of all other functions are 
inherited from Bag and hence do not have to be reimplemented. 


Stack::Stack (int stackCapacity) : Bag(stackCapacity) { } 
# Constructor tor Stack calls constructor for Bag. 


Stack: Stack() { } 


Destructor for Bag is automatically called when Stack 
/1is destroyed. This ensures that array is deleted. 


int Stack::Top() const 
{ 


if (/sEmpty() throw "Stack is empty."; 
return array [top |; 


void Stack::Pop() 
{ 


if (/sEmpty()) throw "Stack is empty. Cannot delete.”; 
fop--} 


} 


Program 3.14: Implementation of Stack operations 


The following code fragment illustrates how inheritance works: 


Bag b(3); # uses Bag constructor to create array of capacity 3 
Stack 9(3}; # uses Stack constructor to create array of capacity 3 
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b. Push(1); &. Push(2); b. Push(3); 
MH use Bag::Push, 


s. Push(\); 5. Push(2) 3s. Push(3); 
Hf Stack::Push not defined, so use Bag::Push. 


b.Pop(); // uses Bag::Pop, which calls Bug::IsEmpty 

5.Pop(); 

Muses Stack::Pop, which calls Bag::JsEmpty because IsEmpty 
has not been redefined in Stack. 


All operations on base class objects (such as & in the example) work in exactly 
the same way they would have, had there been no inheritance. Operations on 
derived class objects (such as s in the example) work differently: if an operation 
is defined in Stack, then that definition is used. If an operation is not defined in 
Stack, then the operation defined in the class that Stack inherits from (Bag in our 
example) is used. After executing the above code, b contains i and 3, while s 
contains | and 2. 
Note that the following also are permissible: 


s.Size(); // uses Bag::Size 
s-Element(); // uses Bag::Element 


If we do not wish to provide access to certain base class functions, then we must 
redefine these functions in the derived class. So, for example, we may redefine 
Element in Stack so as to throw an exception. 

The Queue ADT is also a subtype of Bag. It is more specialized than Bag 
because it requires that elements be deleted in first-in first-out order. Therefore, 
Queue can also be represented as a derived class of Bag. However, there is less 
similarity between the implementations of Bag and Queue than there is between 
Bag and Stack. Like Bag and Stack, Queue uses an array, but unlike Bag and 
Stack, Queue requires the data members front and rear. The implementations of 
the operations /sEmpty, Push, and Pop are different from those of Bag. This 
Means that a larger number of operations need to be redefined in Queue than 
were redefined in Stack. Even though there is not much reuse of implementation, 
there is still reuse of interfaces because the redefined functions of Queue have 
the same prototypes as the original functions in Bag. The exercises explore a 
full implementation of Queue as a derived class of Bag. 
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EXERCISES 


1. Implement Stack as a publicly derived class of Bag using templates. 
Implement Queue as a publicly derived class of Bag using templates. 


A double-ended queue (deque) is a linear list in which additions and dele. 
tions may be made at either end. Implement the class Deque as a publicly 
derived class of Queue. The class Deque must have public functions 
(either via inheritence from Queue or by direct implementation in Deque) 
to add and delete elements from either end of the deque and also to retum 
an element from either end. The complexity of each function (exclusive of 
array doubling) should be ©(1). 

For each of the following pairs state whether they are linked by the IS-A 
relationship. Justify your answer. 

(a) Rectangle and Trapezium. 

(b) Rectangle and Circle. 

(c) Lion and Tiger. 

(d) Stack and Ordered List. 


{e) Queue and Ordered List. 


er 


3.5 AMAZING PROBLEM 


‘The rat in a maze experiment is a classical one from experimental psychology. A 
rat (or mouse) is placed through the door of a large box without a top. Walls are 
set up so that movements in most directions are obstructed. The rat is carefully 
observed by several scientists as it makes its way through the maze until it even- 
tually reaches the exit. There is only one way out, but at the end is a nice chunk 
of cheese. The idea is to run the experiment repeatedly until the rat will zip 
through the maze without taking a single false path. The trials yield its learning 
curve, 

We can write a computer program for getting through a maze, and it will 
probably not be any smarter than the rat on its first try through. It may take many 
false paths before finding the right one. But the computer can remember the 
correct path far better than the rat. On its second try it should be able to go right 
to the end with no false paths taken, so there is no sense re-running the program. 
Why don’t you sit down and try to write this program yourself before you read on 
and look at our solution. Keep track of how many times you have to go back and 
correct something. This may give you an idea of your own learning curve as we 
re-run the experiment throughout the book. 


Let us represent the maze by a two-dimensional array, maze[i] [j], where 
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1 Si <m and 1<j<p. A value of 1 implies a blocked path, and a 0 means one 
can walk right on through. We assume that the rat starts at maze[1}[1], and the 
exit is at maze[m)}[p]. An example is given in Figure 3.11. 


entrance * 


0 
1 
0 
1 
1 
0 
0 
0 
9 
1 
0 


—O- OC OCC ww = CO = 
or ome me =e OO OS 
Oon-ore ee HH ood 
re co-ocoRo- ao 
Se RK Om Ee OS OS me | 
a ei 
HK oCOH Rr moO OI 
Kore rH ooreer a 
COr eee eee eo 
re o--coro-o+| 
-rorHroorroo | 
ee ee ee 
eKe=H2O-SC OF ORK ee 


0 exit 


Figure 3.11: An example maze (can you find a path?) 


With the maze represented as a two-dimensional array, the location of the 
rat in the maze can at any time be described by the row, i, and the column, j, of 
its position. Now let us consider the possible moves the rat can make from a 
point [i] [/] in the maze. Figure 3.12 shows the possible moves from any point {i] 
Lj). The position [i] [j] is marked by an X. If all the surrounding squares have a 
O, then the rat can choose any of these eight squares as its next position. We call 
these eight directions by the names of the points on a compass: north, northeast, 
east, southeast, south, southwest, west, and northwest, or N, NE, E, SE, S, SW, 
‘W, and NW. 

We must be careful here because not every position has eight neighbors. If 
UL] is on a border where either i= 1 or m, or j = 1 or p, then less than eight, and 
possibly only three, neighbors exist. To avoid checking for border conditions, 
we surround the maze by a border of ones. The array will therefore be declared 
as maze[m+2][p +2]. Another device that will simplify the problem is to 
predefine the possible directions to move in a table, move, as in Figure 3.13. The 
data types needed are 
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Figure 3.12: Allowable moves 


struct offsets 


{ 
k 


enum directions { N, NE, E, SE, S, SW, W, NW }5 
offsets move[8] 5 


int a,b; 


If we are at position [é][/] in the maze and we wish to find the position [g][/] that 
is southwest of us, then we set 


g =i + move[SW].a; h = j + move[SW).b; 


For example, if we are at position [3][4], then the position to the southwest is 
given by (3+ 1 =4) [4 +(-1) = 3). 

As we move through the maze, we may have the chance to go in several 
directions. Not knowing which one to choose, we pick one but save our current 
Position and the direction of the last move ina list. This way, if we have taken 4 
false path, we can return and try another direction. With each new location we 
will examine the possibilities, starting from the north and looking clockwise. 
Finally, in order to prevent us from going down the same path twice, we use 
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q movelg).a__move{[q).b 
N -l 0 
NE -1 i 
E 0 1 
SE 1 1 
Ss 1 0 
sw i -l1 
Ww 0 -I 
NW -! -1 


Figure 3.13: Table of moves 


another array, mark({m + 2][p + 2], which is initially zero. mark{i)[/] is set fo 1 
once we arrive at that position. We assume maze(m][p] = 0, since otherwise 
there is no path to the exit. Program 3.15 is a first pass at an algorithm. 


This is not a C++ program, and yet it describes the essential processing 
without too much detai]. The use of indentation for delineating important blocks 
of code plus the use of C++ reserved words make the looping and conditional 
tests transparent. 

What remains to be pinned down? Using the three arrays maze, mark, and 
move, we need only specify how to represent the list of triples (i,j,dir). Since the 
algorithm calls for removing first the most recently entered tiple, this list should 
be a stack. To avoid doubling array capacity during a stack insertion, we need to 
choose a sufficiently large initial capacity for the stack. Since each position in 
the maze is visited at most once, at most mp elements can be placed into the 
stack. Thus, an initial capacity of mp is a safe but somewhat conservative value 
to use for the initial capacity of the stack. The maze of Figure 3.14 has only one 
path from entrance to exit. It has [/2](p-2) + 2 positions. Thus, mp is not too 
crude a bound. We are now ready to give a precise path construction function 
(Program 3.16). 

The arrays maze, mark, and move, are assumed global to Path. Further, 
stack is defined to be a stack of /tems where the struct Items is defined as 


struct tems { 
int x, y, dir; 
oF 
The struct /rems has a constructor (not shown) that sets the values of x, y and dir. 
The operator << is overloaded tor both Stack and Items as shown in Program 
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initialize fist to the maze entrance coordinates and direction east; 
while (list is not empty) 
{ 

(i,j.dir) = coordinates and direction from end of list; 

delete last element of fist; 

while (there are more moves from (i,j)) 


(g,h) = coordinates of next move; 
if ((g==m) && (h == p )) success; 
if ( (Imaze [g J[A]) f legal move 
&& (!mark [g \[h)) // haven’t been here before 


{ 
mark [g \h) = 1; 
dir = next direction to try; 
add (i,j,dir) to end of list; 
‘ G,jdir) = (g,hN); 


} 
cout << “No path in maze.” << endl; 


Program 3.15: First pass at finding a path through a maze 


3.17. We assume that the operator << has been given access to the private data 
members of Stack through the use of the friend declaration. 


Analysis of Path: Now, what can we say about the computing time of the func- 
tion Path? It is interesting that even though the problem is easy to grasp, it is 
difficult to make any but the most trivial statement about the computing time. 
The reason for this is that the number of iterations of the main while loop is 
entirely dependent upon the given maze. What we can say is that each new posi- 
tion {i ][j] that is visited gets marked, so paths are never taken twice. There are 
at most eight iterations of the inner while loop for each marked position. Each 
iteration of the inner while loop takes a fixed amount of time, O(1), and if the 
number of zeros in maze is z, then at most z positions can get marked, Since zis 
bounded above by mp, the computing time is O(enp). (In actual experiments, 
however, the rat may be inspired by the watching psychologists and the invi- 
gorating odor from the cheese at the exit. It might reach its goal by examining 
far fewer paths than those examined by function Path. This may happen despite 
the fact that the rat has no pencil and only a very limited mental stack. It is 
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entrance 


exit 


Figure 3.14: A maze with a long path 


difficult to incorporate the effect of the cheese odor and the cheering of the 
psychologists into a computer program.) Note that the array mark can be elim- 
inated altogether and maze[g ][h] changed to | instead of setting mark([g )[h] to 1, 
but this will destroy the original maze. 0 


EXERCISES 


1, (a) Find a path through the maze of Figure 3.11. 
(b) Trace out the action of function Path (Program 3.16) on the maze of 
Figure 3.11. Compare this to your own attempt in (a). 
2, What is the maximum path length from start to finish for any maze of 
dimensions mm x p? 
3. Write and test recursive version of Path. What is the time complexity of 
your recursive version? 


3.6 EVALUATION OF EXPRESSIONS 


3.6.1 Expressions 


When pioneering computer scientists conceived the idea of higher-level pro- 
gramming languages, they were faced with many hurdles. One of these was how 
to generate machine-language instructions to evaluate an arithmetic expression. 
An assignment statement such as 
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void Path(const int m, const int p) é 
{/ Output a path (if any) in the maze; maze{O}{i) = maze {m +1){i] = 


} 


# start at (1,1) 

mark( LI) = 45 

Stack<ltems> stack(m*p); 

Ttems temp(i, 1, EB); 

# set temp.x, temp.y, and temp.dir 
stack. Push(temp); 

while (!stack.fsEmpty()) 

{// stack not empty 

temp = stack.Top (); 

stack, Pop (); /! unstack 

int i = temp.x; int j = temp.y; int d = temp.dir; 
while (d < 8) // move forward 


) 


{ 


} 


int g = i + move[d].a; int h = j + move{d].b; 
if ((g == m) && (h == p)) { // reached exit 
# output path 
cout << stack; 
cout << i <<" " << j << endl; // last two squares on the path 
cout << m<<"" << p << endl; 
return; 


} 

if ('maze[g][A]) && (!mark[g)[h})) { / new position 
mark(g][h} = 1; 

i; temp.y = j; temp.dir = d+1; 

stack, Push(temp); Hf stack it 

i=; j=h;d=N;/ move to (g,h) 


else d++; // ry next direction 


cout << "No path in maze." << endl; 


Program 3.16: Finding a path through a maze 
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template <class T> 
ostream& operator<<(ostream& os, Stack<T>& s) 


os << “top 
for (int i = 0 ; i <= s.top; i++) 

08 << i <<":" << s.stack[i] << endl; 
return os; 


ostream& operator<<(ostream& os, ltems& item) 


return os << item.x <<"," << itemy <<"," << itemdir; 


Program 3.17; Overloading << 
X =A/B-C+D*E-A*C 


Might have several meanings, and even if it were uniquely defined, say by a full 
use of parentheses, it still seemed a formidable task to generate a correct instruc- 
tion sequence. Fortunately the solution we have today is both elegant and sim- 
ple. Moreover, it is so simple that this aspect of compiler writing is really one of 
the more minor issues. 

An expression is made up of operands, operators, and delimiters. The 
expression above has five operands: A, B, C, D, and E. Though these are all 
one-letter variables, operands can be any legal variable name or constant in our 
programming language. In any expression, the values that variables take must be 
consistent with the operations performed on them. These operations are 
described by the operators. In most programming languages there are several 
kinds of operators that correspond to the different kinds of data a variable can 
hold. First, there are the basic arithmetic operators: plus, minus, times, and 
divide (+, -, *, /). Other arithmetic operators include unary minus, and %. A 
second class is the relational operators: <, ==, <>, >=, and >. These are 
usually defined to work for arithmetic operands, but they can just as easily work 
for character string data. (‘CAT” is less than ‘DOG’ since it precedes ‘DOG’ in 
alphabetical order.) The result of an expression that contains relational operators 
is one of two constants: true or false. Such an expression is called Boolean, 
named after the mathematician George Boole, the father of symbolic logic. 
There also may be logical operators such as &&, ll, and !. 

The first problem with understanding the meaning of an expression is to 
decide in what order the operations are carried out. This means that every 
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language must uniquely define such an order. For instance, if A =4,B =C =2 
D =E = 3, then in the above equation we might want X to be assigned the value. 
((4/2) - 2) + 3 * 3)- (4 * 2) 
=0+9-8 
=} 


However, the true intention of the programmer might have been to assign X the 
value 


(442~2 + 3))* (3-4) #2 
= (4/3) *(-1) *2 
= —2.6666666 


Of course, the programmer could specify the latter order of evaluation by using 
parentheses: 


X =((AAB-C +D))*(E-A)*C 


To fix the order of evaluation, we assign to each operator a priority. Then within 
any pair of parentheses we understand that operators with the highest priority 


will be evaluated first. A set of sample priorities from C++ is given in Figure 
3.15. The highest priority is 1. 


———— 


priority operator 
1 unary minus, ! 
2 *,1,% 
3. +7 
4 PR, > 
5 a 
6 
7 


Figure 3.15: Priority of operators in C++ 


Unary minus and the logical not (!) have top priority, followed by *. /. and 
%. When we have an expression where two adjacent operators have the same 
Priority, we need a rule to tell us which one to perform first. For example, do we 
want the value of A/B*C to be understood as (A/B) * C or AAB*C)? Convince 
yourself that there will be a difference by uying A = B= C=2. The C++ rule is 
that for all priorities, evaluation of operators of the same priority will proceed 
left to right. So, A/B+C will be evaluated as (A/B)* C. Remember that by 
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using parentheses we can override these rules, as expressions are always 
evaluated with the innermost parenthesized expression first. 

Now that we have specified priorities and rules for breaking tices we know 
how X = A/B ~ C + D*E — A*C will be evaluated, namely, as 


X = (((A/B)- C)+(D* E))-(A *C) 
3.6.2 Postfix Notation 


How can a compiler accept an expression and produce correct code? The answer 
is given by reworking the expression into a form we call postfix notation. If e is 
an expression with operators and operands, the conventional way of writing ¢ is 
called infix, because the operators come in-between the operands. (Unary opera- 
tors precede their operand.) The postfix form of an expression calls for cach 
operator to appear after its operands. For example, 


infix A*B/C has postfix AB*C/ 


If we study the postfix form of A*B/C, we see that the multiplication comes 
immediately after its two operands A and B. Now imagine that A*B is computed 
and stored in 7. Then we have the division operator, /, coming immediately after 
its two operands T and C. 

Let us look at our previous example 


infix: A/B-C+D*E~A*C 
postfix: AB/C-DE*+AC*- 


and trace out the meaning of the postfix expression. 

Suppose that every time we compute a value, we store it in the temporary 
location 7;, i 21. If we read the postfix expression left to right, the first opera- 
tion is division. The two operands that precede this are A and B. So, the result of 
A/B is stored in T,, and the postfix expression is modified as in Figure 3.16. 
This figure also gives the remaining sequence of operations. The result is stored 
in T,. Notice that if we had parenthesized the expression, this would change the 
postfix only if the order of normal evaluation were altered. Thus, 
{A/B)—C +(D*E)-A*C will have the same postfix form as the previous 
expression without parentheses. But (A/B}—(C + D)*(E-A)*C will have the 
postfix form AB/CD + EA -—*C*-. 

What are the virtues of postfix notation that enable easy evaluation of 
expressions? To begin with, the need for parentheses is eliminated. Second, the 
priority of the operators is no longer relevant. The expression may be evaluated 
by making a left to right scan, stacking operands, and evaluating operators using 
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operation. postfix 

T, =AB T,C-DE*+AC*— 
T,=T|-C  T,DE*+AC*- 
T3=D*E 12 T3+AC*— 
Ty=T24+T, TAC 

ee ee 
To=Tets Ts 


Figure 3.16: Postfix evaluation 


as operands the correct number from the stack and finally placing the result onto 
the stack (see Program 3.18). This evaluation process is much simpler than 
attempting direct evaluation from infix notation. 


void Eval(Expression e) 
{// Evaluate the postfix expression e. It is assumed that the last token (a token 
//is either an operator, operand, or ’#”) in e is ’#.’ A function NextToken is 
/ used to get the next token from e, The function uses the stack stack 
Stack<Token> stack; i initialize stack 
for (Token x = NextToken(e) ; x !='#'; x = NextToken(e)) 
if (x is an operand) stack.Push(x) // add to stack 
else {// operator 
remove the correct number of operands for operator x from stack; 
perform the operation x and store the result (if any) onto the stack; 


} 


Program 3.18: Evaluating postfix expressions 


3.6.3 Infix to Postfix 


To see how to devise an algorithm for translating from infix to postfix, note that 
the order of the operands in both forms is the same. In fact, it is simple 10 
describe an algorithm for producing postfix from infix: 


(1) Fully parenthesize the expression. 
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(2) Move all operators so that they replace their corresponding right 
parentheses. 


(3) Delete all parentheses. 


For example, A/B-C + D * E-A *C, when fully parenthesized, yields 


((((A/B)-C) +(D*E) )-(A*C)) 
The arcs join an operator and its corresponding right parenthesis. Steps 2 and 3 
yield 


AB/C -DE* +AC #— 


Since the order of the operands is the same in infix and postfix, when we scan an 
expression for the first time, we can form the postfix by immediately passing any 
operands to the output. To handle the operators, we store them in a stack until it 
is time to pass them to the output. 

For example, since we want A+B*C to yield ABC *+, our algorithm 
should perform the following sequence of stacking (these stacks will grow to the 
right): 


nexttoken stack output 


none empty none 
A empty A 

+ + A 

B + AB 


At this point the algorithm must determine if * gets placed on top of the stack or 
if the + gets taken off. Since * has higher priority, we should stack *, producing 


* +8 AB 
C +* ABC 


Now the input expression is exhausted, so we output all remaining operators in 
the stack to get 


ABC #+ 
For another example, A *#(B +C)*D has the postfix form ABC +#D+, and so the 
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algorithm should behave as 


nexttoken stack output 


none empty none 
A empty A 

* * A 

( *( A 

B #( AB 
+ a(t AB 
Cc #(+ ABC 


At this point we want to unstack down to the corresponding left parenthesis and 
then delete the left and right parentheses. This gives us 


) * ABC + 
* = ABC +* 
D * ABC +* D 


done empty ABC+*D* 


These examples motivate a priority-based scheme for stacking and 
unstacking operators. The left parenthesis complicates things, since when it is 
not in the stack, it behaves as an operator with high priority, whereas once it gets 
in, it behaves as one with low priority (no operator other than the matching right 
parenthesis should cause it to get unstacked). We establish two priorities for 
operators: isp (in-stack priority) and icp (in-coming priority). The isp and icp of 
all operators in Figure 3.15 remain unchanged. In addition, we assume that 
isp((*) returns 8, icp(’(’) retuns 0, and isp(*#") returns 8, These priorities result 
in the following rule: Operators are taken out of the stack as long as their in- 
stack priority is numerically less than or equal to the in-coming priority of the 
ne operator. Our function to transform from infix to postfix is given in Program 


Analysis of Postfix: The function makes only a left-to-right pass across the 
input. The time spent on each operand is Q(1). Each operator is stacked and 
unstacked at most once. Hence, the time spent on each operator is also O(1). So 


the complexity of Posifix is Q(n), where n is the number of tokens in the expres- 
sion. 0 
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void Postfix (Expression e) 
{// Output the postfix form of the infix expression e, NextToken 
// is as in function Eval (Program 3.18). It is assumed that 
// the last token in e is ‘#.’ Also, ‘#’ is used at the bottom of the stack 
Stack<Token> stack; // initialize stack 
stack. Push('#'); 
for (Token x = NextToken(e); x !=’#'; x = NextToken(e)) 
if (x is an operand) cout << x; 
else if (x ==’)’) 
{/ unstack until "(" 
for (; stack.Top () !='"('; stack.Pop()) 
cout << stack.Top (); 
stack. Pop (); // unstack ’(’ 


} 
else { // x is an operator 


for (; isp (stack.Top ()) <= icp (x); stack.Pop()) 
cout << stack.Top()3 
stack. Push(x); 


/f end of expression; empty the stack 


for (; !stack.IsEmpty(); cout << stack. Top(), stack.Pop()); 
cout << endl; 


Program 3.19: Converting from infix to postfix form 


EXERCISES 
i. Write the postfix form of the following expressions: 

(a) A*B*C 

(bl) -A+B-C+D 

(c) A*-B+C 

(d) (A+B)*D+EXAF+A*D)+C 

(e) A&& BIICI! (E > F) (assuming C++ precedence) 

(f) 1A &&!(B< CIC > DIC < BE) 
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2. Use the priorities of Figure 3.15 together with those for ‘(" and « . 
answer the following: 


{a) In function Postfix (Program 3.19), what is the maximum number of 
elements that can be on the stack at any time if the input expression ¢ 
has » operators and delimiters? 


(b) What is the answer to (a) if e has operators and the depth of Nesting 
of parentheses is at most 6? 
Another expression form that is easy to evaluate and is parenthesis-free is 


known as prefix. In this way of writing expressions, the operators precede 
their operands. For example: 


infix prefix 
A*BIC FABC 


A/B-C+D*E-A*C  —+ —/ABC *DE* AC 
A*(B+CVYD-G -/* A + BCDG 


Notice that the order of operands is not changed in going from infix to 

prefix. 

(a) What is the prefix form of the expressions in Exercise 1? 

(b) Write a C++ function to evaluate a prefix expression ¢. (Hint: Scan 
é right to left and assume that the leftmost token of e is ‘#.’) 

Write a C++ function to transform an infix expression ¢ into its prefix 


equivalent. Assume that the input expression e begins with a ‘#' and 
that the prefix expression should begin with a ‘#.’ 


{c) 


What is the time complexity of your functions for (b) and (c)? How much 
space is needed by each of these functions? 

Write a C++ function to transform from prefix to postfix. Carefully state 
any assumptions you make regarding the input. How much time and space 
does your function take? 


Do the preceding exercise, but this time for a transformation from postfix 10 
prefix. 


Write a C++ function to generate fully parenthesized infix expressions 


from their postfix form. What is the complexity (time and space) of your 
function? 


Do the preceding exercise starting from prefix form. 
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3.7. ADDITIONAL EXERCISES 


1, [Programming Project] (Landweber] People have spent so much time 
playing card games of solitaire that the gambling casinos are now capitaliz- 
ing on this human weakness. A form of solitaire is described below. Your 
assignment is to write a computer program to play the game, thus freeing 
hours of time for people to return to more useful endeavors. 


To begin the game, 28 cards are dealt into seven piles. The leftmost pile 
has one card, the next two cards, and so forth, up to seven cards in the 
rightmost pile. Only the uppermost card of each of the seven piles is 
turned face up. The cards are dealt left to right, one card to each pile, deal- 
ing to one less pile each time, and tuming the first card in each round face 
up. On the topmost face-up card of each pile you may build in descending 
sequences red on black or black on red. For example, on the 9 of spades 
you may place either the 8 of diamonds or the 8 of hearts. All face-up 
cards on a pile are moved as a unit and may be placed on another pile 
according to the bottommost face-up card. For example, the 7 of clubs on 
the 8 of hearts may be moved as a unit onto the 9 of clubs or the 9 of 
spades. 


‘Whenever a face-down card is uncovered, it is tumed face up. If one pile is 
removed completely, a face-up king may be moved from a pile (together 
with all cards above it) or the top of the waste pile (see below) into the 
vacated space. There are four output piles, one for each suit, and the object 
of the game is to get as many cards as possible into the output piles. Each 
time an ace appears at the top of a pile or the top of the stack, it is moved 
into the appropriate output pile. Cards are added to the output piles in 
sequence, the suit for each pile being determined by the ace on the bottom. 


From the rest of the deck, called the stock, cards are tumed up one 
by one and placed face up on a waste pile. You may always play cards off 
the top of the waste pile, but only one at a time. Begin by moving a card 
from the stock to the top of the waste pile. If there is ever more than one 
possible play to be made, the following order must be observed: 

{a) Move a card from the top of a playing pile or from the top of the 
waste pile to an output pile. If the waste pile becomes empty, move 
a card from the stock to the waste pile. 

({b) Move a card from the top of the waste pile to the leftmost playing 
pile to which it can be moved. If the waste pile becomes empty, 
move a card from the stock to the waste pile. 


{c) Find the leftmost playing pile that can be moved and place it on top 
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of the leftmost playing pile to which it can be moved. 


(a) Try (a), (b), and (c) in sequence, restarting with (a) whenever a Tove 
is made. 


{e) _Ifno move is made via (a) through (d), move a card from the stock 1g 
the waste pile and retry (a). 


Only the topmost card of the playing piles or the waste pile may be played 
to an output pile. Once played on an output pile, a card may not be with. 
drawn to help elsewhere. The game is over when either all the cards have 


been played to the output, or the stock pile has been exhausted and no more 
cards can be moved. 


When played for money, the player pays the house $52 at the beginning 
and wins $5 for every card played to the output piles. Write your program 
so that it will play several games and determine your net winnings. Use a 
random number generator to shuffle the deck. Output a complete record of 
two games in easily understood form. Include as output the number of 
games played and the net winnings (+ or —). 

[Programming Project] {Landweber] We want to simulate an airport land- 
ing and takeoff pattern. The airport has three runways, runway 1, runway 
2, and runway 3. There are four landing holding patterns, two for each of 
the first two runways. Arriving planes will enter one of the holding pattem 
queues, where the queues are to be as close in size as possible. When a 
plane enters a holding queue, it is assigned an integer id number and an 
integer giving the number of time units the plane can remain in the queue 
before it must land (because of low fuel level). There is also a queue for 
takeoffs for each of the three runways. Planes arriving in a takeoff queve 


are also assigned an integer id. The takeoff queues should be kept approxi- 
mately the same size. 


Ateach time, up to three planes may arrive at the landing queues and up to 
three planes may arrive at the takeoff queues. Each runway can handle one 
takeoff or landing at each time slot. Runway 3 is to be used for takeoffs 
except when a plane is low on fuel. At each time unit, planes in either 
landing queue whose air time has reached zero must be given priority over 
other landings and takeoffs. If only one plane is in this category, runway 3 
is to be used. If more than one, then the other runways are also used (at 
each time, at most three planes can be serviced in this way). 


Use successive even (odd) integers for id’s of planes arriving at takeoff 
(anding) queues. At each time unit assume that arriving planes are entered 
into queues before takeoffs or landings occur. Try to design your algorithm 
so that neither landing nor takeoff queues grow excessively. However, 
arriving planes must be placed at the ends of queues. Queues cannot be 
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teordered. 


The output should clearly indicate what occurs at each time unit. Periodi- 
cally output (a) the contents of each queue; (b) the average takeoff waiting 
time; (c) the average landing waiting time; (d) the average flying time 
remaining on landing; and (e) the number of planes landing with no fuel 
reserve. (b) and (c) are for planes that have taken off or landed, respec- 
tively. The output should be self-explanatory and easy to understand (and 
uncluttered). 


The input can be from a terminal, a file, or it can be generated by a random 
number generator. For each time unit the input consists of the number of 
planes arriving at take-off queues, the number of planes arriving at landing 
queues, and the remaining flying times for each plane arriving at a landing 
queue. 


CHAPTER 4 


Linked Lists 


4.1 SINGLY LINKED LISTS AND CHAINS 


In the previous chapters, we studied the representation of simple data structures 
using an array and a sequential mapping. These representations had the property 
that successive nodes of the data object were stored a fixed distance apart. Thus, 
(1) if the element a,; of a table was stored at location Z;;, then aj j4, Was at the 
location Lj; + 1; (2) if the ith element in a queue was at location L,, then the 
(i + 1th element was at location (L; + 1) % n for the circular representation; (3) 
if the topmost node of a stack was at location Ly, then the node beneath it was at 
location Ly - 1, and so on. These sequential storage schemes proved adequate 
for the tasks we wished to perform (accessing an arbitrary node in a table, inser- 
tion or deletion of stack and queue elements), However, when a sequential map- 
ping is used for ordered lists, operations such as insertion and deletion of arbi- 


trary elements become expensive. For example, consider the following list of 
three-letter English words ending in AT: 


(BAT, CAT, EAT, FAT, HAT, JAT, LAT, MAT, OAT, PAT, RAT, SAT, VAT, WAT) 
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To make this list more complete we naturally want to add the word GAT, which 
means gun or revolver. If we are using an array and a sequential mapping to 
keep this list, then the insertion of GAT will require us to move elements already 
in the list either one location higher or lower. We must move either HAT, JAT, 
LAT, «++, WAT or BAT, CAT, EAT, and FAT. If we have to do many such inser- 
tions into the middle, neither alternative is attractive because of the amount of 
data movement. Excessive data movement also is required for deletions. Sup- 
pose we decide to remove the word LAT, which refers to the Latvian monetary 
unit. Then again, we have to move many elements so as to maintain the sequen- 
tial representation of the list. 

An elegant solution to this problem of data movement in sequential 
Tepresentations is achieved by using linked representations. Unlike a sequential 
Tepresentation, in which successive items of a list are located a fixed distance 
apart, in a linked representation these items may be placed anywhere in memory. 
In other words, in a sequential representation the order of elements is the same as 
in the ordered list, whereas in a linked representation these two sequences need 
not be the same. To access list elements in the correct order, with each element 
we store the address or location of the next element in that list. Thus, associated 
with each data item in a linked representation is a pointer or link to the next item. 
In general, a linked list is comprised of nodes; each node has zero or more data 
fields and one or more link or pointer fields. 

Figure 4.1 shows how some of the elements in our list of three-letter words 
may be represented in memory by using pointers. The elements of the list are 
stored in a one-dimensional array called data, but the elements no longer occur 
in sequential order, BAT before CAT before EAT, and so on. Instead we relax 
this restriction and allow them to appear anywhere in the array and in any order, 
To remind us of the real order, a second array, link, is added. The values in this 
array are pointers to elements in the data array. For any i, data{i) and link [i] 
together comprise a node. Since the list starts at data{8] = BAT, let us set a vari- 
able first = 8. link[8] has the value 3, which means it points to data[3}, which 
contains CAT. Since link (3} = 4, the next element, EAT, in the list is in data [4). 
The element after EAT is in data [link [4]]. By continuing in this way we can list 
all the words in the proper order. We recognize that we have come to the end of 
our ordered list when link equals zero. To ensure that a link of zero always 
signifies the end of a list, we do not use position zero of data to store a list ele- 
ment. 

It is customary to draw linked lists as an ordered sequence of nodes with 
links being represented by arrows, as in Figure 4.2. Notice that we do not expli- 
citly put in the values of the pointers but simply draw arrows to indicate they are 
there. The arrows reinforce in our own mind the facts that (1) the nodes do not 
actually reside in sequential focations and (2) the actual locations of nodes are 
immaterial. Therefore, when we write a program that works with lists, we do not 
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wor aunerVUNn 


a2 


Figure 4.1: Nonsequential list representation 


look for a specific address except when we test for zero. The linked structures of 
Figures 4.1 and 4.2 are called singly linked lists or chains. In a singly linked list, 
each node has exactly one pointer field. A chain is a singly linked list that is 
comprised of zero or more nodes. When the number of nodes is zero, the chain 
is empty. The nodes of a non-empty chain are ordered so that the first node links 


to the second node; the second to the third; and so on. The Iast node of a chain 
has a 0 link. 


fit 


[BAT] j—{cat| 4 = WAT] 0] 


Figure 4.2: Usual way to draw a linked list 


Let us now see why it is easier to make insertions and deletions at arbitrary 
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positions using a linked list rather than a sequential list. To insert the data item 
GAT between FAT and HAT, the following steps are adequate: 


(1) Get anode a that is currently unused, 

(2) Set the data field of a to GAT. 

(3) Set the link field of a to point to the node after FAT, which contains HAT. 
(4) Set the link field of the node containing FAT to a. 


Figure 4.3(a) shows how the arrays data and link will be changed after we insert 
GAT. Figure 4.3(b) shows how we can draw the insertion using our arrow nota- 
tion. Dashed arrows are new ones. The important thing to notice is that when 
we insert GAT, we do not have to move any elements that are already in the list. 
We have overcome the need to move data at the expense of the storage needed 
for the field fink, Usually, this penalty is not too severe. When each list element 
is large, significant time is saved by not having to move elements during an 
insert or delete. 

Now suppose we want to delete GAT from the list. All we need to do is 
find the element that immediately precedes GAT, which is FAT, and set /ink[(9] to 
the position of HAT which is 1. Again, there is no need to move the data around. 
Even though the link of GAT still contains a pointer to HAT, GAT is no longer in 
the list as it cannot be reached by starting at the first element of list and follow- 
ing links (see Figure 4.4), 


4.2 REPRESENTING CHAINS IN C++ 


In the following subsections, we shall see the basic techniques used to imple- 
ment chains in C++. In the next section, we generalize the representation using 
templates. 


4.2.1 Defining A Node in C++ 


To define the structure of a node, -we need to know the type of each of its fields. 
The dara field in the example of the previous section is simply a three-character 
string, and the /ink field is a pointer to another node. 

Hf the type of the node in our earlier example is denoted by ThreeLetter- 
Node, then the data type ThreeLetterNode may be defined as a class as follows: 


class ThreeLetterNode { 
private: 

char datal3); 

Three LetterNode *link; 
4 


Exampie 4.1 defines two node structures that can be combined with each 
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first (a) Insert GAT into data [S) 


[BAT] [cat] } ERT 


(b) Insert node GAT into list 


Figure 4.3: Inserting into a linked list 


Figure 4.4: Delete GAT 
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other to form a complex fist structure. 
Example 4.1; The class definitions 


class NodeA { 
private: 
int datal; 
char data2; 
float data3; 
NodeA *linka; 
NodeB *linkb; 
k 
class NodeB { 
private: 
int dara; 
NodeB *link; 
4 
define NodeA to consist of three data fields and two link fields, whereas nodes of 
type NodeB consist of one data field and one link field. Note that the linkb data 
member of nodes of type NodeA must point to nodes of type NodeB. Figure 4.5 
shows a linked list that has one node of each type. 0 


datal : 
data2 
data3 
linka 
linkb 


NodeA NodeB 


Figure 4.5: Illustration of the node structures NodeA and NodeB 
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4.2.2 Designing a Chain Class in C++ 


We have seen how to represent a single node in C++. Now, we shall design a 
C++ representation for a chain of nodes, The basic elements of our design are 
especially important because they apply not just to chains, but also to other 
linked data structures that we shall study later. As is typical with any design pro- 


cess, we shall explore different alternatives, some of which wil! tum out to be 
infeasible or undesirable. 


Design Attempt 1: The variable first, in the ThreeLetterNode example, is 
declared as 


ThreeLetterNode *first; 


The data members of the node pointed to by first may be referenced in the fol- 
lowing way: 


first data, first link 

and the components of data are referenced as 
Sirst~>» data (0), first data (1), first data {2} 
This is shown diagrammatically in Figure 4.6. 


<—<$£_ —_—__——— _ *firss —__—___—_—_"> 


—_\_\_first > data—__—_> 


wo apt 


first dara 0} first—rdara{\} first —»data (2) first link 


Figure 4.6: Referencing the data members of a node 


There is, however, a flaw in our implementation of ThreeLetterNode. 
Notice that the data members data and link are declared as private. This is in 
keeping with the data encapsulation principles discussed in Chapter 1. The 
consequence of declaring the data mambers data and link as private is that the 
expressions first—ink, first data [0], first—rdata {1}, and first—sdata (2] will all 
result in compile-time errors because private data members cannot be accessed 
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from outside the class! 


Design Attempt 2: How do we overcome the drawback of our first attempt? 
Making the data members public would do the trick, but that violates data encap- 
sulation. A solution might be to define public member functions GetLink (), Set- 
Link(), GetData(), and SetData() on ThreeLetterNode which are used to 
indirectly access the data members. This is not a good solution, however, 
because this solution allows one to read and change these data members from 
anywhere in the program. The ideal solution would only permit those functions 
that perform list manipulation operations (like inserting 4 node into or deleting a 
node from a chain) access to the data members of ThreeLetterNode. 


Design Attempt 3: Let us try to tackle this data structure design problem from a 
different perspective: the data structure we are interested in implementing is a 
chain of three-letter words. This suggests that our program should contain a class 
corresponding to the entire chain. Let us call this class ThreeLetterChain. This 
class should contain member functions that carry out list manipulation opera- 
tions such as insert and delete. Thus, we will be using a composite of nwo 
classes, ThreeLetterNode and ThreeLetterChain, to implement our chain. The 
conceptual relationship between the two classes may be characterized by the fol- 
lowing statement: an object of the class ThreeLetterChain consists of zero or 
more objects of the class ThreeLetterNode; or, ThreeLetterChain HAS-A 
ThreeLetterNode. 


Definition: We say that a data object of type A HAS-A data object of type B if A 
conceptually contains B or B is a part of A. For example, Computer HAS-A Pro- 
cessor, Book HAS-A Page, and so on. HAS-A relationships between ADTs are 
usually expressed by making the contained class a member of the containing 
class. This is possible when the type A object contains a fixed number of type B 
objects. O 


Figure 4.7 shows the conceptual relationship between the two classes. Fig- 
ure 4.7 suggests that an object of class ThreeLetterChain physically contains 
many objects of class ThreeLetterNode; that is objects of ThreeLetterNode are 
declared as data members of ThreeLetrerChain. But, the number of nodes in a 
chain is not a constant; on the contrary, it varies with each insert or delete opera- 
tion performed on the list. So, it is impossible to know in advance the number of 
ThreeLetterNodes to include in ThreeLetterChain. For this reason, the ThreeLet- 
terChain class is defined so that it only contains the access pointer, first, which 
Points to the first node in the list. Figure 4.8 shows the actual relationship 
between ThreeLetterChain and ThreeLetterNode objects. It shows that ThreeLet- 
terNode objects are not physically contained inside ThreeLetterChain. The only 
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ThreeLetterChain 


ThreeLetterNode 


je} oar | -}—-feor [+ -- Lar] 


Figure 4.7: Conceptual relationship between 


ThreeLetterChain and 
ThreeLetterNode 


data member contained in ThreeLetterChain is the pointer first. 


ThreeLetterChain 


ThreeLetterNode 


L 


Figure 4.8: Actual relationship between ThreeLetterChain and ThreeLetterNode 


We are now finally ready to propose a solution to our dilemma of how to 
define the class ThreeLetterNode so that only list manipulation operations have 
access to the data members of its nodes, This is achieved by declaring ThreeLet- 
terChain to be a friend of ThreeLetterNode. Only member functions of 
ThreeLetterChain and ThreeLetterNode can now access the private data 


members of ThreeLetterNode. The class definitions of ThreeLetterChain and 
ThreeLetterNode are shown in Program 4.1. 


Nested Classes: An alternative method to represent a chain in C++ is to use 
nested classes, where one class is defined inside the definition of another class. 
Here, class ThreeLetterNode is defined inside the private portion of the definition 
of class ThreeLetterChain, as shown in Program 4.2. This ensures that 
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class ThreeLetterChain; // forward declaration 


class ThreeLetterNode { 
friend class ThreeLetterChain; 
private: 
char data [3)}; 
ThreeLetterNode *link; 
1 


class ThreeLetterChain { 


public: 
#/ Chain manipulation operations 


private: 
ThreeLetterNode * first; 
kb 


Program 4.1: Composite classes 


ThreeLetterNode objects cannot be accessed outside class ThreeLetterChain. 
Notice that the data members of ThreeLeiterNode are public. This ensures that 
they can be accessed by member functions of ThreeLetterChain. Using nested 
classes achieves the same effect as our previous approach, which uses composite 
classes. We use the composite class approach in the rest of the text. One of our 
reasons for preferring composite classes over nested classes is that a single node 
class can be used by many different classes (so long as each is a friend of the 
node class). 


4.2.3 Pointer Manipulation in C++ 


Nodes of a predefined type may be created using the C++ command new. If fis 
of type ThreeLetterNode* then following the call f= new ThreeLetterNode, *f 
denotes the node of type ThreeLetterNode that is created. Similarly, if a and b 
are of type NodeA* and NodeB*, respectively, (Example 4.1) then following the 
execution of the statements 
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class ThreeLetterChain { 
public: 
# Chain Manipulation operations 


private: 
class ThreeLetterNode { // nested class 
public: 
char data [3); 
ThreeLetterNode *link; 
k 
reeLeterNode * first; 


% 


Program 4.2; Nested classes 


a=new NodeA; 
b=new NodeB; 


+a, and *b, respectively, denote the nodes of type NodeA and NodeB that are 
created. These nodes may be deleted in the following way: 


delete f; delete a; delete b; 


C++ also allows pointer variables to be assigned the null pointer constant 0 
(or NULL). This constant is used to denote a pointer that points to no node (€.2., 
the link data member in the last node of Figure 4.3(b)) or an empty list (as in first 
= 0). Addition of integers to pointer variables is permitted in C++ (but is gen- 
erally only used in the context of arrays) . Thus, if x is a pointer variable, the 
expresssion x + I is valid, but may have no logical meaning. Two pointer vari- 
ables of the same type may also be compared to see if both point to the same 
node. Thus, if x and y are pointer variables of the same type, then the expres- 
sions x == y, x != y,x == 0, x != 0 are all valid. The effect of the assignments 
x = y and *x = *y on the initial configuration of Figure 4.9(a) is given in Figure 
4.%(b) and (c). When a pointer is output in C++ using cout, the location in 
memory that the pointer is addressing is output. 
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(a) b)x=y {c) #x = 4y 


Figure 4.9: Effect of pointer assignments 


4.2.4 Chain Manipulation Operations 


In this subsection, we look at three functions that manipulate chains. These 
functions use the classes ChainNode and Chain, where ChainNode is defined as: 


class ChainNode { 
friend class Chain; 
public: 
ChainNode (int element = 0, ChainNode* next = 0) 
0 is the default value for element and next 
{data = element; link = next;} 
private: 
int data; 
ChainNode *link; 
b 


The access pointer first, which points to the first node in the chain, is a private 
data member of Chain. It is assumed that the three functions we are about to 
develop are declared in the public segment of the class definition of Chain. 


Example 4.2: Function Chain ::Create2 (Program 4.3) creates a chain with two 
nodes. The daza field of the first node is set to 10 and that of the second to 20. 
Function Create2 uses the constructor for ChainNode to initialize the fields of 


the two newly created nodes. The resulting list structure is shown in Figure 4.10. 
oa 


Example 4.3: Let first be a pointer to the first node of a chain. first == 0 iff the 
chain is empty (i-e., there are no nodes on the chain). Let x be a pointer to some 
arbitrary node in the chain. Program 4.4 inserts a node with data field 50 
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void Chain ::Create2() 


4 create and set fields of second node 
ChainNode* second = new ChainNode (20,0); 


‘4 create and set fields of first node 
first = new ChainNode (10, second); 
} 


Program 4.3: Creating a two-node list 


frst —o[10 [+20 Jo 
Figure 4.10: A two-node list 


following the node pointed to by x except when the chain is empty. Once again, 
the constructor for ChainNode is used to initialize the fields of the new node. 


The resulting chain structures for the two cases first == 0 and first != 0 are shown 
in Figure 4.11. 0 


void Chain ::insertSO(ChainNode *x) 
{ 


if (frst) 
# insert after x 
x— link = new ChainNode (50, x link); 
else 
insert into empty list 
first = new ChainNode (50); 
} 


Program 4.4: Inserting a node 
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Ts Odeo 


[50[“} 


(a) (b) 


Figure 4.11: Inserting into an empty and nonempty list 


Example 4.4: Let first and x be as in Example 4.3. Let y point to the node (if 
any) that precedes x, and let y == 0 iffx == first. Function Chain ::Delete (Pro- 
gram 4.5) deletes node x from the chain. O 


void Chain ::Delete(ChainNode *x, ChainNode *y) 
{ 

if (x == first) first = first slink; 

else y ~slink = x link; 

delete x; 


} 
Program 4.5: Deleting a node 


EXERCISES 


The following exercises are based on the definitions of Section 4.2.4. All func- 
tions are to be implemented as member functions of Chain and are therefore 
assumed to have access to the data members of ChainNode. 
1. Write a C++ function length to count the number of nodes in a chain. What 
is the time complexity of your function? 
2. Let x be a pointer to an arbitrary node in a chain. Write a C++ function to 
delete this node from the chain. If x == first, then first should be reset to 
point to the new first node in the chain. What is the time complexity of 
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your function? 


3. Write a C++ function to delete every other node beginning with node first 
(ie., the first, third, fifth, and so on nodes of the chain are deleted). What is 
the time complexity of your function? 


4. Letx = (x4,x2, °°°) Xn) and y = (1, ¥2. “++, Ym) be two chains. Write a 
C++ function to merge the two chains together to obtain the chain z= 
(Vt X20 V2e oo KmYorXmats 2s Xn) if mm Sn and z= (x), 94, x2, 
Yo. °° XayYaYass ‘'7s Yn) if m>n. Following the merge, x and y 
should represent empty chains because each node initially in x or y is now 
in z. No additional nodes may be used. What is the time complexity of 
your function? 


5. Let x = (),42, +, 4,) and y = (¥1, 2, ***s Yn) be two chains. Assume 
that in each chain, the nodes are in nondecreasing order of their data-field 
values. Write a C++ function to merge the two chains to obtain a new 
chain z in which the nodes are also in this order. Following the merge, x 
and y should represent empty chains because each node initially in x or y is 
now in z. No additional nodes may be used. What is the time complexity 
of your function? 


6. It is possible to traverse a chain in both directions (i.e., left to right and a 
restricted right-to-left traversal) by reversing the links during the left-to- 


right traversal. A possible configuration under this scheme is given in Fig- 
ure 4.12. Assume that / and r are data members of Chain. 


t r 
oe CeO 


Figure 4.12; Possible configuration for a chain traversed in both directions 


The pointer r points to the node currently being examined and / to the node 

omits teft. Note that all nodes to the left of r have their links reversed. 

(a) Write a C++ function to move pointer r, 2 nodes to the right from any 
given position (/,r). If m is greater than the number of nodes to the 
Fight, set r to 0 and / to the rightmost node in the chain. 


Write a C++ function to move r, n nodes to the left from any given 
Position (/,r). If m is greater than the number of nodes to the left, set / 


(b) 
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to O and r to the leftmost node in the chain. 


43 THE TEMPLATE CLASS CHAIN 


The cost of developing software systems is significant. A contributor is the large 
number of programmer-hours required to develop, test, and maintain software. 
Thus, there is a need for software development strategies that reduce the number 
of person-hours spent without sacrificing the quality of the software. One tech- 
nique for this is software reuse. The basic principle of software reusability is that 
when we initially design and develop software, we must try and do so in a 
manner that makes it possible to reuse the software in the future. This typically 
Tequires a greater investment of time when the software is first developed but 
pays dividends in terms of time saved when the software is reused. 

In this section, we shall enhance the chain class of the previous section so 
that it becomes more reusable. If we are later required to develop software for an 
application that uses chains, we will be able to reuse our enhanced chain class. 


4.3.1 Implementing Chains with Templates 


We have already seen that the template mechanism can be used to make a con- 
tainer class more reusable. A chain is clearly a container class, and is therefore a 
good candidate for implementation with templates. Program 4.6 contains the 
template definition of our enhanced chain class. Notice that Chain<T> has been 
made a friend of ChainNode<T>. This means that the members of 
ChainNode<int> can be accessed by members of Chain<int>, but not by 
members of, say, Chain<float>. Similarly, notice that first is declared to be a 
pointer to an object of type ChainNode<T>. This ensures that an object of type 
Chain<int>, say, only consists of nodes of type ChainNode<int>. 


An empty chain of integers intlist would be defined as: 
Chain<int> intlist; 


4.3.2. Chain Iterators 


An iterator is an object that is used to access the elements of a container class 
one by one. The following discussion motivates the need for iterators of a con- 
tainer class. Consider the following operations that one might wish to perform 
on a container class C, all of whose elements are integers. 
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template <class 7> class Chain; // forward declaration 


template <class T> 
class ChainNode { 
friend class Chain <T>; 
private: 

T data; 

ChainNode <T > *link; 
1 


template <class T> 
class Chain { 
public: 


Chain () {first = 03); // constructor initializing first to 0 
Jf Chain manipulation operations 


private: 
ChainNode <T > *first; 
k 


Program 4.6: Template definition of chains 


(1) Output all integers in C. 
(2) 


(3) 
(4) 


(5) 


Obtain the maximum, minimum, mean, median, or mode of all integers in 
c. 


Obtain the sum, product, or sum of squares of all integers in C. 


Obtain all integers in C that satisfy some property P (for example, P could 
be is a positive integer, is a square of an integer, etc.). 


Obtain the integer x from C such that, for some function f, f (x) is max- 
imum. 


Notice that the solutions to each of these problems require you to examine all 


elements of the container class. The pseudo-code for these problems typically 
takes the following form: 
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initialization step; 
for each item in C 


currentitem = current item of C; 
do something with currentitem; 
} 


postprocessing step; 


For example, to find the maximum of ail elements in the container class, we 
would use the following pseudo-code: 


1 int x = std::numeric_limits<int>::min(); // initialization, must include <limits> 
2 for each item in C 


3{ 

4 currentitem = current item of C; 

5 = x= max(currentlItem, x); 4 do something 

6} 

7 return x; // postprocessing step 


Program 4.7; Pseudo-code for computing maximum element 


The C++ implementation of lines 2 and 4 depends on the container class being 
used, For an array a of size n, the implementation of these lines is: 


2 for (int i= 0; i <n; i++) 
4 currentltem = ali}; 


For a nonempty chain of integers, the implementation of lines 2 and 4 is: 


2 for ( ChainNode<int> *ptr = first; ptr |= 0; ptr = ptrlink) 
4 currenthtem = ptr data; 


Operations such as finding the max element have to be implemented as member 
functions of the particular container class as these operations access private data 
members of the container class. There are some drawbacks to this approach. For 
example, consider the container class Chain <T >. 


(1) Since Chain <T > is a template class, all of its operations should preferably 
be independent of the type of object to which T is instantiated. However, opera- 
tions that are meaningful for one instantiation of T may not be meaningful for 
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another instantiation. For example, computing the sum of all elements in a chain 
makes sense when T = int, but not when T = Rectangle. 


(2) The number of member functions of Chain<T> can become quite large, 
making the class definition rather unwieldy. A class definition should be as com- 
pact as possible so that it is easier for a class user to understand. Moreover, 
classes are often part of a class library designed by one programmer or program- 
ming team. It is not feasible for the class designer to predict all the operations 
required by a particular user of the class. If a user requires an operation that is 
not supported by the class, he or she would be forced to add that operation to the 
container class definition. This is not considered to be a good practice since the 


resulting class definition may not be consistent with the objectives with which 
the original class was designed. 


(3) Even if it is acceptable for the user to add member functions to a class, he or 
she would have to learn how to sequence through the elements of the container 
class, which would entail learning how the container class is implemented. 


These arguments suggest that container classes be equipped with iterators 
that provide systematic access to the elements of the object. Users can employ 
these iterators to implement their own functions depending upon the particular 


application. Typically, an iterator is implemented as a nested class of the con- 
tainer class. 


C++ Iterators 


In C++, an iterator is a pointer to an element of an object (for example, a pointer 
to an element of an array or chain). As the name suggests, an iterator permits you 
to go (or iterate) through the elements of the object one by one. Program 4.8 
shows how to use a pointer (or iterator) y to an array element to iterate through 
the array's elements. The datatype of the pointer y is int*, which indicates that y 
points to elements of type int. In the for loop header, y is initialized to point to 
the first element, x [0], in the array x] (technically, the variable x is a pointer to 
the first element of the array). The expression y++ increments the pointer so that 
it advances to the next element of the array. Similarly, x + 3 is a pointer 3 posi- 
tions from x; that is, it points one position past the last element x [2] of the array. 
So, in the for loop of Program 4.8, y iterates through elements in the range 
lx, x + 3). The expression *y dereferences y so as to get the element pointed to 
by y. The program outputs x [0:2]. 


The following code is equivalent to the for loop of Program 4.8. 
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void main() 
int x [3] = (0, 1,2}; 


// use a pointer y to iterate through the array x 
for (int* y = x; y !=x + 35 y++) 

cout << *y <<" "5 
cout << end]; 


} 


Program 4.8; Using an array iterator 


for (int i = 0; i != 33 i++) 
cout << x[i] <<" "5 
Although you may find this code more transparent than that of Program 4.8, the 


code of Program 4.8 is generalized easily to output the elements of any object for 
which an iterator is defined. For example, the code 


for (Iterator i = start; i != end; i++) 
cout << *i<<" "5 


outputs all elements in the range [start, end). In this code Jterator is the datatype 
of the iterator, start is the iterator value for the first element in the range and end 
is the value the iterator has when incremented one past the last element to be 
output. 

The concept of an iterator is fundamental to writing generic code in C++. 
Program 4.9 gives a possible code for the STL copy function. This code may be 
used to copy elements of any object that has an iterator for which the operators 
‘=, and ++ (postincrement) as well as the dereferencing operator (*) have been 
defined. Different generic codes we write require our iterator to have different 
capabilities. For example, the STL copy_backward function requires us to 
decrement the value of the iterator. : 

To simplify iterator development and categorization of generic iterator- 
based codes, the C++ STL defines five categories of iterators: input, output, for- 
ward, bidirectional and random access. All iterators support the equality opera- 
tors == and != as well as the dereference operator *. Input iterators additionally 
provide read access to the elements pointed at and support the pre- and post- 
increment operator ++. Output iterators provide write access to the elements and 
also permit iterator advancement via the ++ operator. Forward iterators may be 
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template <class /terator> 
void copy (Iterator start, Iterator end, Iterator to) 
{4 copy from [start, end) to [to, to + end — start) 
while (start != end) 
{#10 = *start; start ++; to++;) 


} 


Program 4.9; Possible code for STL copy function 


advanced using the increment operator ++ while bidirectional iterators may be 
incremented as well as decremented (——). Random access iterators are the most 
general. They permit pointer jumps by arbitrary amounts as well as pointer arith- 
metic. C++ array iterators such as y in Program 4.8 are random access iterators. 


A Forward Iterator for Chain 


We may implement a forward iterator class for Chain as in Program 4.10. Our 


implementation requires that Chaintiterator be a public nested member class of 
Chain. 


Additionally, we add the following public member functions to Chain. 
Chainiterator begin() {return Chainlterator (first);} 
Chaintterator end() {return Chainlterator (0);} 


which respectively return iterators initialized to the first node of a list and one 
past the Jast node. We may initialize an iterator object yi to the start of a chain of 
integers y using the statement 

Chain <int>::Chainiterator yi = y.beginQ); 


and we may sum the elements in this chain using the STL algorithm accumulate 
and the statement 


sum = accumulate (y.begin (), y.end (), 0); 


43.3 Chain Operations 


Perhaps the most important decision when designing a reusable class is choosing 
which operations to include. It is important to provide enough operations so that 
the class can be used in many applications. it is also important not to include too 
many operations because this will make the class bulky. Examples of operations 
that would be included in most reusable classes are constructors (including 
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class Chainlterator { 
public: 


‘3 


MH typedefs required by C++ for a forward iterator omitted 


4 constructor 
Chainlterator (ChainNode <T >* startNode = 0) 
{current = startNode;} 


H dereferencing operators 
T& operator *() const {return current —data;} 
T* operator —() const {return &current >data;} 


/ increment 

Chainlterator& operator ++() if preincrement 
{current = current link ; return *this;} 

Chainlterator operator ++(int) // postincrement 


Chainlterator old = *this; 
current = current link, 
return old; 


// equality testing 

boo! operator!=(const Chainlterator right) const 
{return current != right.current;} 

bool operator==(const Chainlterator right) const 
{return current = right.current;} 

private: 
ChainNode <T >* current; 


Program 4.10: A forward iterator for Chain 


default and copy constructors), a destructor, an assignment operator (operators), 
the test-for-equality operator (operator==), and operators to input and output a 
class object (obtained by overloading operator>> and operator<<, respec- 
tively). Other operators may also be overloaded depending on the class. 


A chain class should provide functions to insert and delete elements. There 


could be a number of variations under each of these categories. For example, 
there could be an insertion function that inserts an element at the front of a chain, 
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another that inserts at the end, as in Program 4.11, and yet another that inserts 
after the ith element of the chain. To insert efficiently at the end of a chain, we 


add a private data member last to the class Chain<T>. This data member points 
to the last node in the chain. 


template <class T> 
void Chain <T >::InsertBack(const T& e) 


if (first) {/ nonempty chain 
last ->link = new ChainNode <T >(e); 
last = last link; 


else first = last = new ChainNode <T >(e); 
} 


Program 4.11; Inserting at the back of a list 


Program 4.12 concatenates two chains. Like the function InsertBack, the 
function Concatenate assumes that Chain has been augmented with the private 
data member /ast. The complexity of this function is O(1). 


template <class 7> 
void Chain <T >::Concatenate(Chain <T >& b) 
{// b is concatenated to the end of *this. 
if (first) (last link = b. first; last = b.last;} 
else { first = b.first; last = b.last;} 
b. first = b.last = 0; 


Program 4.12: Concatenating two chains 


Another useful function is one that reverses the order of elements in a chain (Pro- 
gram 4.13). This function is especially interesting because it does an ‘* in- 
place’’ reversal of the elements using three pointers; no element is physically 
moved during the reversal, only pointers are changed. 

_ You should try out Program 4.13 on at least three sample chains, the empty 
chain and chains of length | and 2, to convince yourself that you understand the 
mechanism. For a chain with m 21 nodes, the while loop is executed m times 


Eas va execution of this loop takes O(1) time. So, the computing time is linear 
or my). 
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template <class T> 
void Chain <T >::Reverse () 
{/ A chain is reversed so that (a), *-*, @,) becomes (a,, -**, 41). 


ChainNode <T > *current = first, 
*previous = 03; // previous trails current 
while (current) { 
ChainNode <T > *r = previous; 


previous = current; 4 rails previous 
current = current link; i! current moves to next node 
previous link =r; # link previous to preceding node 


} 


first = previous; 


Program 4.13: Reversing a list 


43.4 Reusing a Class 


Having designed and developed a reusable class for chains. we should now try 
to reuse this class in applications that require chains. In Section 4.7, we illustrate 
how polynomials can be implemented by reusing the chain class. 


Having said that, however, there are some scenarios where one should not 


attempt to reuse a class, but rather design a new one. We present two such 
scenarios below: 


dy 


(2) 


One of the disadvantages of reusing a class (say C,) to implement another 
class (say C2) is that, sometimes, this is less efficient than directly imple- 
menting class C2. If efficiency is of paramount importance, the latter 
approach is usually the better course of action. In Section 4.6, we use 
chains to implement stacks and queues. Since each is an important data 
structure, which will itself be reused in other applications, we implement 
them directly to make each as efficient as possible. 

Another scenario in which classes should not be reused is when the opera- 
tions required by the application are complex and specialized, and there- 
fore not offered by the class. In this situation, also, it is preferable to 
develop the application directly rather than to “force” the application to 
reuse an existing class. In Section 4.8, we use chains to solve the problem 
of finding equivalence classes. The chain operations required by our algo- 
rithin are specialized and not likely to be implemented in a reusable chain 
class. Hence, we implement equivalence classes directly. 
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EXERCISES 
The following exercises assume that chains are defined as in Program 4.6. All 


funct 
L 


44 


ions are to be implemented by employing Chainiterator to traverse a chain. 


Write a C++ template function to output all elements of a chain by over- 
loading the output operator <<. Assume that the operator << has been over- 
loaded for the data type T. 


Write a C++ function to compute the minimum of all elements of a chain of 
integers. 


Write a C++ function to copy the elements of a chain into an array. Use the 
STL function copy together with array and chain iterators. 


Let.x),x2, ***, X, be the elements of a chain. Each x; is an integer. Write 
n= 
a C++ function to compute the expression > (x;*x;45). [Hint: use two 


isl 
iterators to simultaneously traverse the chain.} 
Fully code and test the C++ template class Chain<T>. You must include 
a constructor, which constructs an initially empty chain; a destructor, 
which deletes all nodes in the chain; functions to insert at either end of the 
chain; functions Front and Back that, respectively, return the first and last 
elements of the list; a function Get (i) that returns the ith element in the list; 
functions to delete from either end; functions to insert as the ith element 
and to delete the ith element, and a forward iterator. 


CIRCULAR LISTS 


A circular list (or, more precisely, a singly-linked circular list) may be obtained 
by modifying a chain so that the fink field of the last node points to the first node. 
Figure 4.13 shows how the chain of Figure 4.2 is modified to form a circular list. 


Sirs 
\ 


e 


S sar | {car | }—{ gar [+ --. +f war] | 


| 
L 


f 


Figure 4.13: A circular list 
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The circular list structure has some advantages over the chain (as we shaft 
see later). The C++ implementation of circular lists is very similar to that for 
chains except for some minor differences: To check whether a pointer current 
points to the last node of a circular list, we check for (current link == first) 
instead of (current link == 0). Secondly, the functions for insertion into and 
deletion from a circular list must ensure that the dink field of the last node points 
to the first node of the list upon completion. 

Let us take a look at an operation on a circular list. Suppose we want to 
insert a new node at the front of the list of Figure 4.14. We have to change the 
link of the node containing x3, which requires that we move down the entire 
length of the list until we find the last node. It is more convenient if the access 
pointer of a circular list points to the last node rather than to the first (Figure 
4.15). 


first GSS er 


data link 


Figure 4.14; Example of a circular list 


x1] -4 =[ 2X2 = k fast 


data link 


Figure 4.15: Pointing to the last node of a circular list 


Now we can write functions that insert at the front (Program 4.14) or at the 
back of a circular list in O(1) time. To insert e at the back, one only needs to add 
the statement last = newNode to the else clause of InsertFront. The code of Pro- 
gram 4. 14 assumes the existence of the template class CircularList similar to the 
class Chain of Program 4.6. It is assumed that CircularList contains the private 
data member fast that points to the last node of the list and that CircularList is a 
friend of ChainNode. In some applications, using the structure of Figure 4.13 
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template <class T> 
void CircularList <T >::InsertFront(const T& e) 
{i Insert the element ¢ at the ‘‘front’” of the circular 
iH Nist *this, where last points to the last node in the fist. 
ChainNode <T > *newNode = new ChainNode <T >(e); 
if (/asz) { // nonempty list 
newNode->link = last->link; 
last link = newNode; 


} 
else { // empty list 
last = newNode; 
newNode ~link = newNode; 


} 


Program 4.14: Inserting at the front of a circular list 


causes problems as the empty list has to be handled as a special case. To avoid 
such special cases, we introduce a dummy node, called the header node, into 
each circular list (i.c., each instance of a circular list will contain one additional 
node). Thus, the empty list will have the representation of Figure 4.16(a) and the 


list of Figure 4.13 will have the representation of Figure 4.16(b). 


head “Pick 7] 


(a) 


>{Bat| |—{car|} {ear f= —>{war 


(b) 


Figure 4.16: Circular lists with a header node 
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EXERCISES 


Do Exercise 1 of Section 4.2 for the case of circularly linked lists. 
Do Exercise 2 of Section 4.2 for the case of circularly linked lists. 
Do Exercise 3 of Section 4.2 for the case of circularly linked lists. 
Do Exercise 4 of Section 4.2 for the case of circularly linked lists. 
Do Exercise 5 of Section 4.2 for the case of circularly linked lists. 
Do Exercise 6 of Section 4.2 for the case of circularly linked lists. 
Do Exercise 1 of Section 4.3 for the case of circularly linked lists. 
Do Exercise 2 of Section 4.3 for the case of circularly linked lists. 
Do Exercise 3 of Section 4.3 for the case of circularly linked lists. 
Do Exercise 4 of Section 4.3 for the case of circularly linked lists. 


Repeat the previous exercise for circularly linked lists with a header node. 
This time, assume that CircularListWithHeader contains a private data 
member, header, that points to the header node. 


FSerrnanaun- 


4.5 AVAILABLE SPACE LISTS 


The destructors for chains and circular lists take time linear in the length of the 
chain or linear list as they delete nodes one at a time. The run time of these des- 
tructors may be reduced to O(1) if we maintain our own chain of free (or deleted) 
nodes. When a new node is needed, we may examine our chain of free nodes. If 
this chain is not empty, then one of the nodes on it may be made available for 
ba Only when this chain is empty do we need to invoke new to create a new 
node. 

Consider the class CircularList of Section 4.4. Let av be a static class 
member of CircularList<T> of type ChainNode<T>* that points to the first 
nade in our chain of nodes that have been ‘‘deleted.”” (We implement the chain 
of deleted nodes directly rather than reusing our class Chain <T > for efficiency 
reasons.) Our chain of deleted nodes will henceforth be called the available- 
Space list or av list. Initially, av = 0. Instead of using the functions new and 
delete, we shall now use the functions CircularList <T>::GetNode (Program 
4.15) and CircularList <T >::RetNode (Program 4.16). 

__ As illustrated by function CircularList <T >:2CircularList (Program 4.17), 
a circular list may now be deleted in a fixed amount of time independent of the 
number of nodes on the list. Figure 4.17 is a schematic showing the link changes 
involved in deleting a circular list. 


A chain may be deleted in O(1) time if we know its first and last nodes. 
The instructions: 
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template <class T> 
ChainNode <T >* CircularList <T >::GetNode() 
{i Provide a node for use. 

ChainNode <T >* x3 

if (av) {x = av; av = av—link;} 

else x = new ChainNode <T >; 

return x; 


} 


Program 4.15: Getting a node 


template <class T> 
void CircularList <T >::RetNode(ChainNode <T >*& x) 
{i/ Free the node pointed to by x. 

xlink = av; 

av=Xx; 

x=0; 


} 


Program 4,16: Returning a node 


template <class KeyT> 
void CircularList <T >::"CircularList () 
{4 Delete the circular list. 
if (last) { 
ChainNode <T >* first = last— link; 


last~ link = av; # last node linked to av 
av = first, 
last = 0; 


} 


4 first node of list becomes front of av list 


Program 4.17: Deleting a circular list 
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last 


Figure 4.17: Dashed arrows indicate changes involved in deleting a circular list 


last—>link = av; 
av = first; 


accomplish this. When we have our own available space list, we may do other 
tasks quickly as well. For example, we can generalize the GetNode function so 
that GetNode (n) returns a chain or circular list with 2 nodes. Since the available 
space list already has most of the links correctly set, we can reduce the time 
spent setting links when creating chains and circular lists of known length. 


46 LINKED STACKS AND QUEUES 


We have already seen how to represent stacks and queues sequentially. In this 
section we examine how to use linked lists to represent stacks and queues. Fig- 
ure 4.18 shows a linked stack and a linked queue. 

Notice that the direction of links for both the stack and the queue is such 
that it facilitates insertion and deletion of nodes. In the case of Figure 4.18(a), 
you can easily add a node at the top or delete one from the top. In Figure 
4,18(b), you can easily add a node at the rear, and both addition and deletion can 
be performed at the front, although for a queue we do not want to add nodes at 
the front. The public members of the linked stack and queue classes LinkedStack 
and LinkedQueue are, respectively, the same as those given in the stack and 
queue ADTs, ADT 3.1 and ADT 3.2. LinkedStack has the single private data 
member top, which points to the top node of the stack and LinkedQueue has two 
private data members frent and rear, which, respectively, point to the first and 
last nodes of the queue. Both classes use nodes of type ChainNode <T> (Sec- 
tion 4.3) and it is assumed that both classes are declared as friends of 
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(b) Linked queue 


(a) Linked stack 


Figure 4.18: Linked stack and queue 


ChainNode <T>. The constructor for LinkedStack sets top to 0 (NULL) and that 
for LinkedQueue sets both front and rear to 0. The codes for the ADT functions 
IsEmpty, Top, Front and Rear are quite similar to those for the case when an 
array was used to represent a stack and a queue. Functions for insertion and 
deletion from linked stacks and queues are presented in Programs 4.19 - 4.22. 


template <class T> 
void LinkedStack <T >::Push(const T& e) { 
top = new ChainNode <T >(e, top); 


Program 4.19: Adding to a linked stack 


Linked Stacks and Queues 201 


template <class 7> 
void LinkedStack <T >::Pop() 
{// Delete top node from the stack. 
if (/sEmpty ()) throw "Stack is empty. Cannot delete.”; 
ChainNode <T > *delNode = top; 
top =top—link; — // remove top node 
delete delNode; —_// free the node 
} 


Program 4.20: Deleting from a linked stack 


template <class T> 
void LinkedQueue <T >::Push(const T& e) 


if (/sEmpty ()) front = rear = new ChainNode(e, 0); // empty queue 
else rear = rear link = new ChainNode(e, 0); // attach node and update rear 


Program 4.21: Adding to a linked queue 


template <class T> 

void LinkedQueue <T >::Pop() 

{// Delete first element in queue. 
if (isEmpty ()) throw "Queue is empty. Cannot delete.”; 
ChainNode <T > *delNode = front; 
front = front link; — // remove first node from chain 
delete delNode; // free the node 


Program 4.22: Deleting from a linked queue 
EXERCISES 


i. Develop and test a complete C++ template class for linked stacks. 
2. Develop and test a complete C++ template class for linked queues. 


3. Consider the hypothetical data object X2. X2 is a linear list with the res- 
triction that although insertions to the list may be made at either end, dele- 
tions can be made from ane end only. Design a linked-list representation 
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for X2. Develop and test a complete C++ template class implementation 
for X2. 


Develop and test a C++ template class that implements the queue ADT 
using a circularly linked list. 


Implement the stack data structure as a derived class of the class List <T> 
of Section 4.3. Test your code. 


Implement the queue data structure as a derived class of the class List <T> 
of Section 4.3. Test your code. 


4.7 POLYNOMIALS 


4.7.1 Polynomial Representation 


Let us tackle a reasonable-sized problem using linked lists. This problem, the 
manipulation of symbolic polynomials, has become a classic example of the use 
of list processing. In general, we want to represent the polynomial 
a(x) = a,x" + +++ +a)x°', where the a; are nonzero coefficients and the 
exponents e, are nonnegative integers such that &m > @m—) > °'' >€2 >, 20. 
We shall define a Polynomial class to implement polynomials. Since a polyno- 


mial is to be represented by a list, we have Polynomial IS-IMPLEMENTED-BY 
List. 


Definition: We say that a data object of Type A IS-IMPLEMENTED-IN- 
TERMS-OF a data object of Type B if the Type B object is central to the imple- 
mentation of the Type A object. This relationship is usually expressed by declar- 
ing the Type B object as a data member of the Type A object. 0 


Thus, we shall make the chain object poly a data member of Polynomial. For 
this, we employ the chain template class of Program 4.6. Each ChainNode will 
represent a term in the polynomial. To do this, the list template 7 is instantiated 
to struct Term, where, Term consists of two data members coef and exp. We use 
struct rather than class to define Term to emphasize that the data members of 
Term are public. The data members of Term are made public so that any function 
that has access to a Term object also has access to its data members. This does 
not violate data encapsulation for Polynomial because the linked list and its con- 
tents, including all Term objects are all private and cannot be accessed. Assum- 
ing that the coefficients are integers, the required class declarations are 
developed in Program 4.23. 

Note that ChainNodes contain two fields data and link, and, since data is of 


type Term, data contains two fields coef and exp. For clarity, we draw the nodes 
in a polynomial as below. 
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struct Term 
{(// All members of Term are public by default, 
int coef; —// coefficient 
int exp; 4 exponent 
Term Set(int c, int e) {coef = c; exp = e} return *this;); 


1 


class Polynomial { 
public: 
4H public functions defined here 
private: 
} Chain <Term > poly; 
3 


Program 4,23: Polynomial class definition 


coef [exp | link | 


Figure 4.19 shows the representation for the polynomials a = 3x!4 + 2x8 +1 
and b = 8x'4 — 3x'0 + 10x°. 


a poly, first ——2[3 [14] -} $278] +-{1] 070] 
(a) 

b.poly. first [3 [i4y 4 {-3 [10] }—a{ 10] 6] 0 
(b) 


Figure 4.19: Representation of 3x'4+2x8 +1 and 8x!4—3x 9+ 10x 
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4.7.2 Adding Polynomials 


To add two polynomials a and 6, we use the list iterators ai and bi to examine 
their terms starting at the nodes pointed to by a first and b first. If the exponents 
of two terms are equal, then the coefficients are added. A new term is created for 
the result if the sum of the coefficients is not zero. If the exponent of the current 
term in a is less than the exponent of the current term in 6, then a copy of the 
term in b is created and attached to the Polynomial object c. The iterator ib is 
advanced to the next term in 6. Similar action is taken on a if 
ai~ exp > bi~exp. Figure 4.20 illustrates this addition process on the polyno- 
mials @ and b of the example of Figure 4.19. 

Each time a new node is created, its coef and exp data members are set and 
the node is appended to the end of c by function InsertBack (Program 4.11). The 
complete addition code is specified by the function operator+() (Program 4.24). 

This is our first complete example of the use of list processing, so it should 
be carefully studied. The basic algorithm is straightforward, using a merging 
process that streams along the two polynomials, either copying terms or adding 
them to the result. Thus, the main while loop of lines 7-21 has three cases 
depending upon whether the exponents of the terms are ==, <, or >. 


Analysis of operator+{): The following tasks contribute to the computing time: 
(1) coefficient additions 


(2) exponent comparisons 
(3) _ inserting a term at the end of a chain. 


Let us assume that each of these three tasks, if done once, takes a single unit of 
time. The total time taken by operator+() is then determined by the number of 
limes these tasks are performed. This number clearly depends on how many 


terms are present in the polynomials a (*this) and b. Assume that a and b have 
mand n terms, respectively: 


(x) = yx 4 Fax”, BOX) = by + bx" 
where 


a,b, #0 and e, > --- >e,20,f,>-..>f;20 
Then clearly the number of coefficient additions can vary as 
OS coefficient additions < min{m,n) 


The lower bound is achieved when none of the exponents are equal; the upper 
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Figure 4,20: Generating the first three terms of c = a +b 


bound is achieved when the exponents of one polynomial are a subset of the 
exponents of the other polynomial. 

As for exponent comparisons, one comparison is made on each iteration of 
the first while loop. On each iteration either ai or bi or both move to the next 
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1 Polynomial Polynomial::operator+(const Polynomial& b) const 
2 {#/ Polynomials *this (a) and b are added and the sum retumed. 
ai Term temp} 

Chain <Term >::Chainlterator ai = poly.begin (), 


bi = b.poly.begin (); 


while (ai && bi) { // current nodes are not null 


4 
5 
6 Polynomial c; 
7 
8 if (ai Sexp == bi exp)) { 


9 int sum = ai coef + bi >coef; 
10 if (sum) c.poly.InsertBack (temp.Set (sum,ai >exp)); 
WW ait++; bi++; // advance to next term 
12 } 
13 else if (ai exp < bi-vexp) { 
14 c.poly.InsertBack (temp.Set (bi coef, bi >exp)); 
15 bi ++; // next term of b 
16 } 
ie] else { 
18 c.poly.InsertBack (temp.Set (ai coef , ai >exp)); 
19 ai ++; // next term of a 
20 } 
21 } 
22 while (ai) {// copy rest of a 
23 c.poly.InsertBack (temp. Set (ai >coef , ai exp)); 
24 ai++; 
25 } 
26 while (bi) {/ copy rest of b 
27 c.poly.InsertBack (temp.Set (bi coef , biexp))s 
28 bits 
29 } 
30 return c; 
31} 


Program 4.24: Adding two polynomials 


term in their respective polynomials. Since the total number of terms is m +7, 
the number of iterations and hence the number of exponent comparisons is 


bounded by m + n. You can easily construct a case when m + n — | comparisons 
will be necessary —e.g.,m =n and 


en > fa > Cnt > fr > 0 > er > fp > ey >f 
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The maximum number of terms in ¢ is m +n, so no more than m +n nonzero 
terms are inserted at the end of c. In summary, the maximum number of execu- 
tions of any of the statements in operator+() is bounded above by m +n. There- 
fore, the computing time is O(m + 1). Since any algorithm that adds two polyno- 
mials must look at each nonzero term at least once, our code for operator+() is 
optimal to within a constant factor. 0 


4.73 Circular List Representation of Polynomials 


Figure 4.21 shows the circular list representation of a polynomial. The structure 
of Figure 4.21, however, causes some problems during addition and other poly- 
nomial operations, as the zero polynomial has to be handled as a special case. 
To avoid such special cases we introduce a header node into each polynomial 
(i.e., each polynomial, zero or nonzero, will contain one additional node). The 
exp and coef data members of this node will not be relevant. Thus, the zero poly- 
nomial will have the representation of Figure 4.22(a), and a = 3x!4 + 2x5 +1 
will have the representation of Figure 4.22(b). 


Figure 4.23: Circular list representation of 3x4 + 2x® +1 


Using this circular list with header representation, the addition algorithm 
takes the form given in Program 4.25. This algorithm assumes that the exp field 
of the header node of a polynomial is -1 and that the begin () function for Circu- 
larListWithHeader returns an iterator that points to the node head link. Now 
when all terms of a (*this) have been examined, ai is at the header and 
ai exp =-1. Since -] < bi—exp, the remaining terms of b can be copied by 
further executions of the while loop. The same is true if all terms of b are exam- 
ined before those of a. This implies that there is no need for additional code to 
copy the remaining terms as in Program 4.24. 
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head —>{ os 


(a) Zero polynomial 


head 


\f 
CTS Ble ee soap 


(by 3x!4 4 2x8 41 


Figure 4.22: Example polynomials 


EXERCISES 


When a class object is passed by value, it is copied into the function's local 
store. The algorithm for copying the object is specified by the copy con- 
structor. If the object’s class definition does not define the copy construc- 
tor, the default copy constructor is used. The default copy constructor only 
copies the data members of the object. Define a copy constructor 


Polynomial(const Polynomial& p) 


that copies all the terms of the polynomial p into *this. Do not forget to 


delete all terms that may be in *this prior to the invocation of the copy con- 
structor. 


Overload the C++ input operator << so that objects of type Polynomial 
may be input using this operator. Your code should read in n pairs of 
coefficients and exponents, (c;,¢;), | Si <n, of a univariate polynomial and 
convert the polynomial into the circularly linked list representation 
described in this section. Assume that e; >¢;,;, 1S$i<n, and that 
c, #0, 1 $i Sa. Show that this operation can be performed in time O(n). 

Let a and b be two polynomials represented as circular lists with header 
nodes. Write a C++ function to compute the product polynomial c = a*b. 
Your code should leave a and b unaltered. Show that if n and m are the 
number of terms in a and 6, respectively, then this multiplication can be 


carried out in time O(nm?) or O(nn2), If a and b are dense, show that the 
multiplication takes O(n). 
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1 Polynomial Polynomial::operator+(const Polynomial& b) const 
2 {4 Polynomials *this (a) and 6 are added and the sum retumed. 


3 Term temp; 
4 CircularListWithHeader <Term >::Iterator ai = poly.begin (), 
5 bi = b.poly.begin 0; 
6 Polynomial c; // assume constructor sets head exp = —1 
7 while (1) { 
8 if (ai mexp = biexp)) { 
9 if (ai exp == -1) return c; 
10 int sum = ai coef + bi coef; 
11 if (sum) c.poly.InsertBack (temp.Set (sum,ai exp); 
12: ai++; bi++; / advance to next term 
13 
14 else if (ai exp < bi >exp) { 
15 c.poly.InsertBack (temp.Set (bi coef, bi—exp)); 
16 bi++; // next term of b 
17 
18 else { 
19 c.poly.InsertBack (temp.Set (ai coef , ai exp)); 
20 ai++3// next term of a 
21 } 
22 
23} 
Program 4.25: Adding two polynomials represented as circular lists with header 
nodes 
4, 


Write a C++ function to evaluate a polynomial at the point x, where x is a 
real number. Assume that the polynomial is represented as a circularly 
linked list with a header node. 


{Programming Project| Develop a C++ class Polynomial to represent and 
manipulate univariate polynomials with integer coefficients (use circular 
linked lists with header nodes). Each term of the polynomial will be 
represented as a node. Thus, a node in this system will have three data 
members as below: 
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ieoef | exp | link 


Each polynomial is to be represented as a circular list with header node, 
To delete polynomials efficiently, we need to use an available-space list 
and associated functions as described in Section 4.5. The extemal (i.e., for 
input or output) representation of a univariate polynomial will be assumed 
to be a sequence of integers of the form: n,c), €1, C2, €2, €3, €3, 
++ Cny @n, Where e; represents an exponent and c; a coefficient; n gives 
the number of terms in the polynomial. The exponents are in decreasing 
order—e, >€,> **' > ey. 

‘Write and test the following functions: 


(a) istream& operator>>(istream& is, Polynomiat& x): Read in an 


input polynomial and convert it to its circular list representation 
using a header node. 


{b) ostream& operator<<(ostream& os, Polynomial& x): Convert x 
from its linked list representation to its external representation and 
output it. 

{c) Polynomial ::Polynomial(const Polynomial& a) [Copy Constructor]: 
Initialize the polynomial «this to the polynomial a. 

(@) const Polynomial& Polynomial::operator=(const Polynomial& a) 
const [Assignment Operator]: Assign polynomial a to *this. 

(©) Polynomial::"Polynomial() [Destructor]: Return all nodes of the 
polynomial *this to the available-space list. 

® 


Polynomial operator+ (const Polynomial& b) const [Addition]: 

Create and return the polynomial *this + b. 

(g) Polynomial operator— (const Polynomial& b) const (Subtraction) : 
Create and return the polynomial «this — b. 

(h) Polynomial operator+(const Polynomial& b) const (Multiplication): 

Create and return the polynomial *this * b. 

(i) float Polynomial ::Evaluate(Hoat x) const: Evaluate the polynomial 

*this at x and return the result. 


48 EQUIVALENCE CLASSES 


Let us put together some of the ideas discussed so far on linked and sequential 
representations to solve a problem that arises in the design and manufacture of 
very large-scale integrated (VLSI) circuits. One of the steps in the manufacture 
of a VLSI circuit involves exposing a silicon wafer using a series of masks. Each 
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mask consists of several polygons. Polygons that overlap are electrically 
equivalent. Electrical equivalence specifies a relationship among mask 
polygons. This relation has several properties that it shares with other relations, 
such as the conventional mathematical equivalence. Suppose we denote an arbi- 
trary relation by the symbol =, and suppose that 


(1) For any potygon x, x =x (e.g.,.x is electrically equivalent to itself). Thus, = 
is reflexive. 

(2) For any two polygons x and y, if x =y, then y =x. Thus, the relation = is 
symmetric. 


(3) For any three polygons x, y, and z, if x=y and y #2, then xz (e.g., if x 
and y are electrically equivalent and y and z are also, then so also are x and 
2). The relation = is transitive. 


Definition: A relation = over a set S, is said to be an equivalence relation over S 
iff it is symmetric, reflexive, and transitive over S. O 


Equivalence relations are numerous. For example, the ‘‘equal to’’ (=) relation- 
ship is an equivalence relation, since (1) x = x, (2) x = y implies y = x, and (3) 
x =y and y =z implies x =z. One effect of an equivalence relation is to parti- 
tion the set S$ into equivalence classes such that two members x and y of S are in 
the same equivalence class iff x = y. For example, if we have 12 polygons num- 
bered 0 through 11 and the following overlap pairs are defined: 
0=4,3=1,6=10, 8=9, 7=4, 6=8, 3=5, 2= 11, and 1120 
then, as a result of the reflexivity, symmetry, and transitivity of the relation =, we 
get the following partitioning of the 12 polygons into three equivalence classes: 
(0, 2,4, 7, 11}5 (1,3, 5}; (6,8, 9, 10} 
These equivalence classes are important, as each equivalence class defines a sig- 
nal net. The signal nets can be used to verify the correctness of the masks. 

Our algorithm to determine equivalence classes works in essentially two 
phases. In the first phase the equivalence pairs (i,j) are read in and stored. In 
phase two we begin at 0 and find all pairs of the form (0, J). The values 0 and j 
are in the same class. By transitivity, all pairs of the form (j, k) imply & is in the 
same class as 0. We continue in this way until the entire equivalence class con- 
taining 0 has been found and output. Then we find an object not yet output. This 
is in a new equivalence class. The objects in this equivalence class are found as 
before and output. 

The first design for our equivalence class algorithm might be as in Program 
4.26. Let » and n represent the number of input pairs and the number of objects, 
respectively. We need to select a data structure to hold these pairs. Easy random 
access would suggest a Boolean array, say pairs(n|{n]. The element pairs[i][j] = 
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void Equivalence() 
{ 
initialize; 
while more pairs 
{ 
input the next pair (i,j); 
process this pair, 


initialize for output; 
for (each object not yet output) 
output the equivalence class that contains this object; 


Program 4.26: First version of equivalence algorithm 


true if and only if (i,j) is an input pair. However, using such an array could 
potentially be quite wasteful of space since very few of the array elements may 
be used. Any algorithm that uses this data structure would require O(n”) time, 
just to initialize the array. 

These considerations lead us to consider a chain to represent each row of 
the aforementioned array pairs. Each node on a chain requires only a data and a 
link field. However, we still need random access to the ith row. For this, we use 
a one-dimensional array, first{n], with first{i] a pointer to the first node in the 
chain for row i. 

Looking at the second phase of the algorithm we need a mechanism that 
tells us whether or not object j is yet to be printed. A Boolean array, out {nJ, can 
be used for this. Program 4.27 gives the next refinement of our equivalence algo- 
rithm. 


Let us simulate the algorithm, as we have it so far, on the input data set 
0=4,3=1,6210,8=9, 7=4, 628, 3=5,2=11l,and1150 
After the while loop is completed the chains look as in Figure 4.23. For each 
relation i = j, two nodes are used. first{i} points to a chain of nodes that con- 
tains every number directly equivalent to i by an input relation. 

In phase two, we scan the first array in the order i, OS i <n. For each f 
such that owt [i} is false, the elements in the list first |i] are output. To enable the 
Processing of the remaining elements which, by transitivity, belong in the same 
Class as i, a linked stack is created. Thus, a node may initially be on one of the 
chains given by first(] and later be on a linked stack of elements waiting to be 
processed. The complete function is given in Program 4.28. 


Equivalence Classes 213 


void Equivalence() 

{ 
read n; # read in number of objects 
initialize first (0:n-1) to 0 and out [0:n—1) to false; 
while more pairs // input pairs 


read the next pair (i,j); 
put j on the chain first {i]; 
put i on the chain first [j]; 
for (i= 03 i< nj i++) 
if (lout [i]) { 
out [i] = true; 
output the equivalence class that contains object i; 


} 


Program 4.27: A more detailed version of equivalence algorithm 


frss (0) (1) [2] B) (4) (5) (6 (7) (8) (9) HO} fy) 


data 
link 


data 
link 


Figure 4.23: Lists after pairs have been input 
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class ENode { 
friend void Equivalence (); 
public: 
ENode(int d = 0) / constructor 
{data = d; link = 03} 


private: 
int data; 
ENode *link; 
‘ 


void Equivalence () 

{// Input the equivalence pairs and output the equivalence classes. 
ifstream inFile( “equiv.in", ios::in); // "equiv.in" is the input file 
if (!inFile) throw “Cannot open input file.”; 
int i, j, 23 
inFile >> n; // read number of objects 
// initialize first and out 
ENode *%first = new ENode* [n}; 
bool “out = new bool [7]; 

4 use STL function fill to initialize 
fill (first, first +n, 0); 
fill(out, out +n, false); 


H Phase 1: input equivalence pairs 

inFile >> i >> j5 

while (inFile.good ()){ // check end of file 
first [i] =new ENode (j, first {i}; 
first i) = new ENode (i, first); 
inFile >>i>> j; 


} 


i! Phase 2: output equivalence classes 
for (§ = 05 i <n; i++) 
if (!out [i ]) ( // needs to be output 
cout << end! << "A new class: "<< i; 
out [i] = true 5 
ENode *x = first [i]; ENode *top = 0; // initialize stack 
while (1) { // find rest of class 
while (x) ( // process the list 
j=a-data; 
if (lour{j]) { 
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cout <<", "<< j; 
out {j]= true; 
ENode *y =x link; 
x~link = top; 

fop =x; 

x=Y3 


else x = x link; 
} 4 end of while(x) 
if (top) break; 
x= first [top >data |; 
top = top link; // unstack 
}/ end of while(1) 
} / end of if (tout [i}) 
for (i = 03 i <n; i++) 
while (first[iJ) { 
ENode *deinode = first{i); 
first{i) = delnode—link; 
delete deinode; 


} 
; delete [ } first; delete [ ] out; 


Program 4.28: C++ function to find equivalence classes 


Analysis of Equivalence: The initialization of first and out takes O(n) time. 
The processing of each input pair in phase | takes a constant amount of time. 
Hence, the total time for this phase is O(n + m), where m is the number of input 
pairs. In phase 2 each node is put onto the linked stack at most once. Since 
there are only 2m nodes and the for loop is executed n times, the time for this 
phase is O(m +). Hence, the overall computing time is O(m +). Any algo- 
rithm that processes equivalence relations must look at ali m equivalence pairs 
and at all # objects at least once. Thus, no algorithm can have a computing time 
less than ©(m + 2). This means that function Equivalence is optimal to within a 
constant factor. Unfortunately, the space required by the algorithm is also 
O(m + 1). In Chapter 5 we shall see an O(n)-space solution to this problem. 


EXERCISES 


1. Rewrite function Equivalence (Program 4.28) using an array of objects of 
type Chain <int> rather than an array of type ENode*. Also, use an object 
of type Stack <int> (see Section 3.2) rather than the custom linked stack 
used in Program 4.28. What can you say about the expected relative 
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performance of your function and Program 4.287 Which requires more 
space? 


Rewrite function Equivalence (Program 4.28) using dynamic arrays and 
array doubling in place of chains (see Sections 2.5 and 3.1,1). Compare the 
worst-case space and time complexities of the two versions of Equivalence. 


4.9 SPARSE MATRICES 


4.9.1 Sparse Matrix Representation 


In Chapter 2, we saw that when matrices were sparse (i.e., many of the entries 
were zero), then much space and computing time could be saved if only the 
nonzero terms were retained explicitly. In the case where these nonzero terms 
did not form any ‘‘nice’’ pattern such as a triangle or a band, we devised a 
sequential scheme in which each nonzero element was represented by a structure 
with three data members: row, column, and value. The sequential representation 
of Chapter 2 permits easy access of matrix terms by row. However, accessing all 
the terms in a specific column of a matrix is difficult. To provide easy access 
both by row and by column, we devise a linked representation for a sparse 
matrix. In the data representation we use, each nonzero element is in two circu- 
lar lists; one is a row list and the other a column list. So, we have a circular list 
for each row and each column of the matrix. Each circular list has a header 
node. 

Our sparse matrix representation uses nodes of the type MatrixNode. This 
class has a Boolean field head, which is used to distinguish between header 
nodes and nodes that represent nonzero elements. Each header node has three 
additional fields: down, right, and next. The total number of header nodes is 
max (nuimber of rows, number of columns}. The header node for row i is also 
the header node for column i. The down field of a header node is used to link 
into a column list; the right field is used to link into a row list. The next field 
links the header nodes together. 

All other nodes have six fields: head, row, col, value, down, and right (Fig- 
ure 4.24). The down field is used to link to the next nonzero term in the same 
column and the right field links to the next nonzero term in the same row. Thus, 
if a; #0, then there is a node with head = false, value = aij, row = i, and col =j. 
This node is linked into the circular linked lists for row i and column j. Hence, 
the node is simultaneously in two different lists. 

As noted earlier, each header node is in three lists: a row list, a column list, 
and a list of header nodes. The list of header nodes itself has a header node, H, 
that is identical to the nodes used to represent nonzero elements. The row and 
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(a) header node (b) element node 
head field is not shown 


Figure 4.24; Node structure for sparse matrices 


col fields of this node are used to store the matrix dimensions. The entire matrix 
is represented by class Matrix with the data member headnode, which points to 
H. 
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Figure 4.25: 5 x 4 sparse matrix a 


The linked structure obtained for the 5x4 matrix, a, of Figure 4.25 is 
shown in Figure 4.26. Although Figure 4.26 does not show the values of the 
head fields, these values are readily determined from the node structure shown, 
For each nonzero term of a, we have one six-field node that is in exactly one 
column list and one row list. The header nodes are marked HO to H4 and are 
drawn twice to simplify the figure. As seen in the figure, the right field of head- 
node is used to link into the list of header nodes. Notice that the whole matrix 
may be referenced through the header node, headnode, of the list of header 
nodes. 

If we wish to represent an n Xm sparse matrix with r nonzero terms, the 
number of nodes needed is max{n,m} +r + 1. Although each node may require 
several words of memory, the total storage needed will be less than nm for 
sufficiently small r. 

The required node structure may be defined in C++ using an anonymous 
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Figure 4.26: Linked representation of the sparse matrix of Figure 4.25 (the head 
field of a node is not shown) 


union as in Program 4.29. Since all nodes contain fields head, down and right 
these are declared outside the anonymous union. Header nodes also contain nex 
while all other nodes contain row, col, and value. These fields ace all ai 
porated into the struct Triple. Space is allocated by the union declaration for 

larger of the fields next and triple (which is an object of type Triple). 
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struct Triple {int row, col, value;}; 
class Matrix; // forward declaration 
class MatrixNode { 
friend class Matrix; 
friend istream& operator>>(istream&, Matrix&); f for reading in a matrix 
private: 
MatrixNode *down , *right; 
bool head; 
union { // anonymous union 
MatrixNode *next; 
Triple triple; 
ki 
MatrixNode(bool, Triple +); // constructor 
5 


MatrixNode::MatrixNode(bool b, Triple * t) // constructor 
{ 


head = b; 
if (6) {right = down = this;} // row/column header node 
else triple = +t; // element node or header node for list of header nodes 


class Matrix { 
friend istream& operator>>(istream&, Matrix&); 
public: 
~MatrixQ; / destructor 
private: 
MatrixNode *headnode; 
5 


Program 4.29: Class definitions for sparse matrices 


4.9.2 Sparse Matrix Input 


The first operation we shall consider is that of reading in a sparse matrix and 
obtaining its linked representation. We assume that the first input line gives us 
the number of rows, the number of columns, and the number of nonzero terms in 
the matrix. Each subsequent input line is a triple of the form (i, j, a),). These tri- 
ples consist of the row, col, and value data members of the nonzero terms of the 


matrix. We also assume that these triples are ordered by rows and within rows 
by columns. 
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For example, the input for the 5 x 4 sparse matrix of Figure 4.25, which has 
six nonzero terms, would take the following form: 5, 4, 6; 0, 0, 2; 1,0, 4; 1,3,3: 
3, 0, 8; 3, 3, 1; 4, 2,6. The code for the input operator >> (Program 4.30) makes 
use of an auxiliary array head, of size max(s.row, s.col). head [i], which is a 
pointer to the header node for column i and hence also for row i, enables efficient 
random access to columns while the input matrix is set up. Function operator>> 
first creates the header nodes (but doesn’t link them together) and then inputs the 
matrix elements one at a time. Each element is added to its row and column lists 
upon input. The next field of header node i is used initially to keep track of the 


last node in column i. Eventually, in fine 31, the header nodes are linked 
together through this field. 


Analysis of operator>>: Assuming that new works in O(1) time, all the header 
nodes may be set up in O(max{n,nt}) time, where n is the number of rows and m 
the number of columns in the matrix being input. Each nonzero term is set up in 
(1) time because of the use of the variable last and a random access scheme for 
the bottommost node in each column list. Hence, the for loop of tines 15-26 
takes O(r) time. The rest of the algorithm takes O{max({n,m}) time. The total 
time is therefore O(max{nym} +r) = O(n +m+r)). Note that this time is 
asymptotically better than the input time of O(nm) for an n x m matrix using a 


two-dimensional array but slightly worse than that for the sequential sparse 
method of Section 2.3, O 


4.9.3 Deleting a Sparse Matrix 


All the nodes of a sparse matrix may be retumed one at a time using delete. A 
faster way to return the nodes is to set up an available-space list (Section 4.5). 
Assume that av points to the first node of this list and that this list is linked 


through the field right. Function Matrix::"Matrix() (Program 4.31) deletes 4 
Sparse matrix in an efficient way. 


Analysis of “Matrix(): Since each node is in only one row list, it is sufficient to 
return all the row lists of the matrix. Each row list is circularly linked through 
the field right. Thus, nodes need not be retumed one by one, as a circular list can 
be deleted in O(1) time. The computing time for Program 4.31 is readily seen (0 


be O(n +m). Note that even if the available-space list had been linked through 
the field down, deleting still could have been carried out in O(n + m) time. 0 
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| istream& operator>>(istream& is, Matrix& matrix) 

2 {// Read in a matrix and set up its linked representation. 

3 Triple s; 

4 is >> s.row >> $.col >> s.value; // matrix dimensions 

5 int p= max(s.row, s.col); 

6 # set up header node for list of header nodes. 

7 matrix.headnode = new MatrixNode (false, &s); 

8 if (p ==0) { matrix.headnode right = matrix.headnode; return is;} 
9 at least one row or column 

10 MatrixNode **head = new MatrixNode * (p)}; 

JI] for (int i = 05 i <p; i++) 

12 head {i] = new MatrixNode(true, 0); 

13. int currentRow =0; 

14 int MatrixNode *last = head {0); // last node in current row 


15 for (i = 0; i <s.value; i++) // input criples 


{ 
17 Triplet; 
18 is >>trow >> t.col >> t.value; 
19 if (t.row > currentRow) { // close current row 
20 last >right = head [currentRow }; 
21 currentRow = t. row; 
22 last = head [currentRow ); 
23. -} Wend of if 
24 = last = last right = new MatrixNode(false, &t); // Sink new node into row list 
25 head [t.col]—»next = head [t.col)~»next >down = last; if link into column list 
26 }// end of for 


27 last right = head |currentRow }; /! close last row 

28 for (i =03 i <s.col; i++) head [i }—next >down = head (i); // close all column lists 
29 if \ink the header nodes together 

30 for (i = 0; i< p—1; i++) head [i next = head [i +1); 

31 head [p-1]—next = matrix.headnode; 

32 matrix.headnode right = head (0); 

33 delete [ | head; 

34 return is; 

35} 


Program 4.30: Reading in a sparse matrix 


222 Linked Lists 


Matrix:: Matrix Q) 
{/ Return all nodes to the av list. This list is a chain linked via the right 
H field. av is a static variable that points to the first node of the av list, 


if (theadnode) return; // no nodes to delete 
MatrixNode *x = headnode right; 
headnode —right = av; av = headnode; // retumn headnode 
while (x != headnode) { // erase by rows 
MatrixNode *y =x right; 
x right = av; 
av=y; 
x=x—next; // next row 


} 
headnode = 0; 


—)s 


Program 4.31; Deleting a sparse matrix 


EXERCISES 


In Exercises 1 to 5, the sparse matrix representation described in this section is 
assumed. 


1. 


Write the C++ function, operator+(const Matrix& b) const, which retums 
the matrix *this + b. Show that if *this and b are n x m matrices with a 
and r, nonzero terms, then this addition can be carried out in 
O(n +m + ry + r,) time. 

Write the C++ function, operator*(const Matrix& b) const, which returns 
the matrix *this + b. If @ is an m x m matrix with r, nonzero terms and if 
“this is an n xm matrix with r, nonzero terms and b is an m xp matrix 
with ry nonzero terms, then this multiplication can be done in O(pra + 9) 
fae Can you think of a way to do the multiplication in O(min{pra» nrel) 
ume? 


Write the C++ function operator<<( ), which outputs a sparse matrix as (rl 
ples (i,j,a;;). The triples are to be output by rows and within rows by 
columns. Show that this operation can be performed in time O(n +" a) if 
there are 7, nonzero terms in the matrix. n is the number of rows 11 the 
matrix. 


4. Write the C++ function, Transpose(), which transposes a sparse matnx 


What is the computing time of your function? 
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Write and test a copy constructor for sparse matrices. What is the comput- 
ing time of your copy constructor? 


(Programming Project] A simpler and more efficient representation for 
sparse matrices can be obtained when one is limited to the operations of 
addition, subtraction, and multiplication. Now, nodes have the data 
members rowLink, colLink, row, col, and value. Each nonzero term is 
represented by a node. These nodes are linked together to form two circu- 
lar lists. The first list, the row list, is made up by linking nodes by rows and 
within rows by columns. The linking is done via the right data member. 
The second list, the column list, is made up by linking nodes via the down 
data member. In this list, nodes are linked by columns and within columns 
by rows. These two lists share a common header node. In addition, a node 
is added to contain the dimensions of the matrix. Draw the resulting 
representation for the matrix of Figure 4.25. 


Write a C++ class for this representation. You must include the following 
functions. What is the computing time of each of your functions? How do 
these times compare with the corresponding times for the representation of 
this section? 


(a) istream& operator>>(istream& os, Matrix& m): 

Read in the matrix and set it up according to the representation of 
this section. The first input line gives the matrix dimensions. The 
next several Jines contain one triple. (row, column, value), each. The 
last triple ends the input file. These triples are in increasing order by 
rows. Within rows, the triples are in increasing order of columns. 
The data is to be read in one line at a time and converted to internal 
representation. 


(b) ostream& operator<<(ostream& as, const Matrix& m): 
Output the terms of m. To do this, you will have to design a suitable 
output format. The output should be ordered by rows and within 
rows by columns. 
(c) Matrix ::Matrix(const Matrix& a) [Copy Constructor]: 
Initialize the sparse matrix *this to the sparse matrix a. 
(d) const Matrix& Matrix::operator=(const Matrix& a) const [Assign- 
ment Operator]: Assign sparse matrix a to *this. 
(e)  Matrix:: Matrix () [Destructor]: 
Return all nodes of the sparse matrix *this to the available-space list. 
(f) Matrix Matrix :operator+(const Matrix& b) const: 
Create and return the sparse matrix *this + 5. 


(g) Matrix Matrix :-operator—(const Marrix& b) const: 
Create and return the sparse matrix *this — b. 
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(bh) Matrix Matrix :operator* (const Matrix& b) const: 
Create and retum the sparse matrix *this * b. 
(i) Matrix Matrix ::‘Transpose () () const: 

Create and return the transpose of *this, 


7. Compare the sparse representations of Exercise 6 and this section with 
respect to the time needed to output the elements in an arbitrary row or 
column. 

8. 


{Programming Project] Implement a complete linked-list system to per- 
form arithmetic on sparse matrices using the representation of this section, 
You must include all the functions listed in Exercise 6. 


4.10 DOUBLY LINKED LISTS 


So far we have been working chiefly with chains and singly linked circular lists. 
For some problems these would be too restrictive. One difficulty with these lists 
is that if we are pointing to a specific node, say p, then we can move only in the 
direction of the links. The only way to find the node that precedes p is to start at 
the beginning of the list. The same problem arises when one wishes to delete an 
arbitrary node from a singly linked list. As can be seen from Example 4.4, easy. 
deletion of an arbitrary node requires knowing the preceding node. If we have a 
problem in which it is necessary to move in either direction or in which we must 
delete arbitrary nodes, then it is useful to have doubly linked lists. Each node 
now has two link fields, one linking in the forward direction and the other linking 
in the backward direction. 

A node in a doubly linked list has at least two fields—e.g., left (left link), 
and right (right link). Usually, each node will have a data field as well. A dou- 
bly linked list may or may not be circular and may or may not have a header 
node. A sample doubly linked circular list with header node is shown in Figure 
4.27. This list has a header node plus three additional nodes. As was true in the 
earlier sections, header nodes are convenient for the algorithms. Now suppose 
that p points to any node in a doubly linked list. So, p == pleftright == 
poright—ieft. This formula reflects the essential virtue of this structure— 
namely, that one can go back and forth with equal ease. When header nodes are 
used, an empty list has exactly one node—the header node (Figure 4.28). Pro- 
gram 4.32 contains the class definition of a doubly linked list of integers. To 
work with these lists we must be able to insert and delete nodes. Function 
DbIList:-Detete (Program 4.33) deletes node x from the list. Following the dele- 
tion, x points to a node that is no longer Part of the list. Figure 4.29 shows how 
the function works on a doubly linked list with only a single node. Even though 
the right and feft data members of node x still point to the header node, this node 
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Header Node 


Figure 4.27: Doubly linked circular list with header node 


Peet ea 


Figure 4.28: Empty doubly linked circular list with header node 


has effectively been removed, as there is no way to access x through first. Inser- 
tion is only slightly more complex (see Program 4.34). 


EXERCISES 


1. Let x be a node in a singly linked circular list. Write a C++ function to 
delete the data in this node. Following the deletion, the number of nodes in 
the list is one less than before the deletion. Your function must run in O(1) 
time. (Hint: Instead of deleting the node x, copy the data from the node, if 
any, that is next to x and delete that node instead.) 

2. Write a function void DbiList::Concatenate(DbiList m) to concatenate the 
two lists *this and m. On completion of the function, the resulting list 


should be stored in *this and the list m should contain the empty list. Your 
function must run in O(1) time. 
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class DbiList; 


class DbIListNode { 
friend class DbiLisr; 
private: 

int data; 

DbIListNode *left, *right; 
b 


class DbiList { 
public: 
/f List manipulation operations 


private: 


DblListNode * first; // points to header node 
1 


Program 4.32: Class definition of a doubly linked list 


void DbiList ::Delete(DbIListNode *x) 


else { 
x left—sright = x right; 
x Srightleft = x left; 
delete x; 
} 
} 


if (x == first) throw "Deletion of header node not permitted"; 


Program 4.33: Deletion from a doubly linked circular list 


3. Devise a linked representation for a list in which insertions and deletions 


can be made at either end in O(1) time. Such a structure is called a deque. 
Write functions to insert and delete at either end. 
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Before After 
first —>4 : first 


x ——>| x 


Figure 4.29: Deletion from a doubly linked circular list 


void DbIList ::Insert(DbiListNode *p, DblListNode *x) 
{// insert node p to the right of node x 

pleft =x; pright = x— right; 

x ~rightleft = p; x right = p; 


Program 4.34: Insertion into a doubly linked circular list 


4. Consider the operation XOR (exclusive OR, also written as ®) defined as 
follows (for i, j binary): 


- @ ;— |0 ifi and j are identical 
eB y= ( otherwise 


This definition differs from the usual OR of logic, which is defined as 


0 ifi=j=0 


FORI=1) otherwise 


The definition can be extended to the case in which i and j are binary 
strings (i.e., take the XOR of corresponding bits of i and j). So, for exam- 
ple, if (= 10110 and j = 01100, then i XOR j = /®j= 11010. Note that 


a@(a@b)=(Ga)Ob=d 
and 
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(a@b)@b=a@b@b)=a 


This notation gives us a space-saving device for storing the right and left 
links of a doubly linked list. The nodes will now have only two data 
members: data and link. If | is to the left of node x and r to its right, then 
x~ link = 1 @® rIf xis the leftmost node of a non-circular list, / = 0, and if x 
is the rightmost node, r= 0. For a new doubly linked list class in which the 


link field of each node is the exclusive or of the addresses of the nodes to 
its left and right, do the following. 


(a) Write a C++ function to traverse the doubly linked list from left to 
Tight, printing out the contents of the dara field of each node. 

(b) Write a C++ function to traverse the list from right to left, printing 
out the contents of the data field of each node. 

(Programming Project]: Implement a C++ template class for doubly 

linked circular lists with header nodes. You must include a constructor, 


copy constructor and destructor as well as functions to insert and delete. A 
bidirectonal iterator must be included as well. 


4.11 GENERALIZED LISTS 


4.11.1 Representation of Generalized Lists 


In Chapter 2 a linear list was defined to be a finite sequence of n 2 0 elements, 
Qo, **',@_-1, Which we write as (ao, «++, @,-1). The elements of a linear list 
are restricted to atoms; thus, the only structural property a linear list has is that of 
position (i.e., a; precedes a;,;,0 <i <n-—1). Relaxing this restriction on the ele- 
ments of a list and permitting them to have a structure of their own leads to the 


notion of a generalized list. Now, the elements a), 0<i <n—1, may be either 
atoms or lists. 


Definition: A generalized list, is a finite sequence of n 20 elements, 


dy. °° dy), Where a; is either an atom or a list. The elements a,, 0 <i Sn-l, 
that are not atoms are said to be the sublists. O 


Let A = (ag, +++, a1) be a list. A is the name of the list (ay, °° "+ @n-1): 
and 1 is its length. By convention, all list names are represented by capital 
levers. Lowercase letters are used to represent atoms. If n > 1, then ao is the 


head of A, and (a), ---.a,.,) is the tail of A. Some examples of generalized 
fists are 


{1)A = (4: the null, or empty, list; ts length is zero. 


(2) B = (a.(b.c)): a list of length two; its first element is the atom a, and its 
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second element is the linear list (b,c). 


(3) C = (B,B, 0): a list of length three whose first two elements are the list B, and 
the third element is the null list. 


(4) D = (a,D): a recursive list of length two; D corresponds to the infinite list 
(a, (a, (a, +++). 


A is the empty list. For list B, we have head(B) = ‘a’ and tail(B) = ((b,¢)); 
tail (B) also has a head and tail, which are (b,c) and (), respectively. Looking at 
list C, we see that head(C) = B and tail(C)=(B,()). Continuing, we have 
head (tail (C)) = B and tail (rail (C)) = (()), both of which are lists. 

Two important consequences of our definition for a list are (1) lists may be 
shared by other lists as in example (3), where list B makes up two of the sublists 
of C; and (2) lists may be recursive as in example (4). The implications of these 
two consequences for the data structures needed to represent lists will become 
evident. 

First, let us restrict ourselves to the situation in which the lists being 
represented are neither shared nor recursive. To see where this notion of a list 
may be useful, consider the problem of representing polynomials in several vari- 
ables. For example, 


P(x,y,2) = x!y32? + 2x8y3z2? + 3x8y2z? + x4y4z + Grd y'z + 2yz 


Each term of P may be represented using a structure that has four fields: 
coef, expx, expy, and expz. Going this route would require us to define one struc- 
ture for polynomials in one variable, another for those in two variables, and so 
on. Alternatively, we could define a structure with a large number of exponent 
fields and use only as many as needed. Neither of these solutions is elegant. 

Tf we rewrite P (x,y,z) as 


(Gx'9 + 2x8 yy? + BxBy2hc? + (a4 + 6x yt + 2y)z 


we sce that P(x,y,x) may be viewed as a generalized list. Every polynomial can 
be written in this fashion, factoring out a main variable z, followed by a second 
variable y, and so on. Looking carefully now at P(x,y,z), we see that there are 
two terms in the variable z, Cz? and Dz, where C and D are polynomials them- 
selves, but in the variables x and y. Looking more closely at C(x,y), we see that 
it is of the form Ey? + Fy?, where E and F are polynomials in x. Continuing in 
this way, we see that every polynomial consists of a variable plus coefficient- 
exponent pairs. Each coefficient is itself a polynomial (in one less variable) if we 
regard a single numerical coetticient as a polynomial in zero variables. 

We see that every polynomial, regardless of the number of variables in it, 
can be represented using nodes of the type PolyNode, defined as 
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enum Triple {var, ptr, no}; 
class PolyNode 
it 


PolyNode *next; # link field 

int exp; 

Triple trio; 

union { 
char vile; 
PolyNode *down; If link field 
int coef; 

+H 

b 


In this representation, there are three types of nodes, depending on the value of 
trio. If trio == var, then the node is the header node for a list; in this case, the 
field vble is used to indicate the name of the variable on which that list is based 
and the exp field is set to 0. (Note that the type of the data member vble can be 
changed to int if all variables are kept in a table and vble just gives the 
corresponding table index.) If trio == ptr, then the coefficient is itself a list and 
is pointed by the field down. If trio == no, then the coefficient is an integer and is 
stored in the field coef. In the last two cases, exp represents the exponent of the 
variable on which that list is based. 

Figure 4.30 gives the representation of the polynomial 3xy. Here first 
points to a header node, which indicates that the upper list is based on y. Hence 
the next term in the list refers to the term y! = y. The coefficient of this term is 
the polynomial represented by the lower list. This polynomial is 3x?, Figure 
4.31 gives the representation of the polynomial P (x,y,z) defined earlier. For sim- 


plicity, the rio field is omitted from Figure 4.31. The value of this field for each 
node is self-evident. 


triv_vble exp next trio down exp next 


peed y[o rel, 1 [o | 

I 

| Ivar | x. 0 }+[no 13 2) 0) 
first 


Figure 4.30: Representation of 3x7y 


Every generalized list can be represented using the node structure 
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zo HE 3 nog 


S 


ing 


Figure 4.31: ((x! + 2x®)y? + 3x8y2)z? + (@4 + 6x3)y? + 2y)z 


This data structure may be defined in C++ as: 
template <class T> class GenList; // forward declaration 


template <class 7> 
class GenListNode { 
friend class GenList <T>; 
private: 
GenListNode <T > *next; 
bool tag; 
union { 
T data; 
GenListNode <T > *down; 


}; 


template <class T> 
class GenList { 
public: 
// List manipulation operations 


private: 
GenListNode <T > * first; 
3 
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The data/down field holds an atom if head (A) is an atom and holds a pointer to 
the list representation of head(A) if head (A) is a list. (The exercises examine 
how a mulitvariate polynomial may be stored using this structure.) Using this 
node structure, the example lists (1) to (4) (page 228) have the representation 
shown in Figure 4.32. 


afirst=0 empty list 


bfirst 
B =(a, (b,c) 


D=(a,D) 


ee era a ee a 
Figure 4.32: Representation of lists (1) to (4) (page 228); an f in the tag field 
represents the value false, whereas a t represents the value true) 


4.11.2 Recursive Algorithms for Lists 


When a data object is defined recursively, it is often easy to describe recursive 
algorithms that work on these objects. Recursive algorithms for C++ objects 
typically consist of two components—the recursive function itself (the wor- 
khorse) and a second function that invokes the recursive function at the top level 


(the driver ). The driver is declared as a public member function while the wor- 
khorse is declared as a private member function. 
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4.1.2.1 Copying a List 


To see how recursion is useful, let us write a function (Program 4.35) that pro- 
duces an exact copy of a nonrecursive list in which no sublists are shared. 


# Driver 
void GenList <T >::Copy(const GenList <T >& 1) const 
{4 Make a copy of I. 
first = Copy (L first); 
) 


4 Workhorse 
GenListNode <T >* GenList <T >::Copy(GenListNode <T >* p) 
{// Copy the nonrecursive list with no shared sublists pointed at by p. 
GenListNode <T > *q = 0; 
if@){ 
q = new GenListNode <T >; 
q tag = pag; 
if (p tag) g >down = Copy (p down); 
else g data = p data; 
gq —next = Copy (p next); 
} 
return g; 


} 
Program 4.35: Copying a list 


Program 4.35 reflects exactly the definition of a (generalized) list. We see 
immediately that Copy works correctly for an empty list. A simple proof using 
induction will verify the correctness of the entire function. Now let us consider 
the computing time of this function. The empty list takes a constant amount of 
time. For the list A = ((@,b),((c,d),e)), which has the representation of Figure 
4.33, p takes on the values given in Figure 4.34. The sequence of values should 
be read down the columns; 6, r, s, t, u, v, w, and x are the addresses of the eight 
nodes of the list. From this example one should be able to see that nodes with 
tag = false will be visited twice, whereas nodes with tag = true will be visited 
three times. Thus, if a list has a total of m nodes, no more than 3m executions of 
any statement will occur. Hence, the algorithm is O(m), or linear, which is the 
best we can hope to achieve. Another factor of interest is the maximum depth of 
recursion or, equivalently, how many locations are needed for the recursion 
stack. Again, by carefully following the algorithm on the previous example, we 
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see that the maximum depth is a combination of the lengths and depths of all 
sublists. However, a simple upper bound is m, the total number of nodes. 
Although this bound will be extremely large in many cases, it is achievable, for 
instance, if A = (((((a))))). 


first 


Figure 4.33: Linked representation for A 


level of continuing continuing 

recursion value of p level Pp level Pp 
1 b 2; C 3 u 

Z s 3 u 4 v 

3 t 4 w 5 0 

4 0 5 x 4 v 

3 t 6 0 3 u 

2 s $s x 2 f 

1 b 4 w 3 0 

2. t 

1 b 


Figure 4.34 Values of parameters in execution of GenList <T >::Copy (A) 
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4.11.2.2 List Equality 


Another useful function determines whether two lists are identical. To be identi- 
cal, the lists must have the same structure and the same data in corresponding 
data members. Again, using the recursive definition of a list, we can write a 
short recursive function (Program 4.36) to accomplish this task. 


HU Driver 
template <class T> 
bool operator==(const GenList <T >& 1) const 
{// *this and / are non-recursive lists. 
4 The function returns true iff the two lists are identical. 
return Equal (first, first); 


M4 Workhorse 
bool Equal(GenListNode <T >* s, GenListNode <T> *t) 


if ((!s) && (4s) return true; 
if (s && t && (s stag == t1ag)) 
if (s tag) 
return Equal(s down, t down) && Equal(s next, t-snext); 
else return (s data == t data) && Equal (s next, t next); 
return false; 


Program 4.36: Determining if two lists are identical 


The computing time of Program 4.36 is clearly no more than linear when 
no sublists are shared, since it looks at each node of the two lists being compared 
no more than three times. For unequal lists the program terminates as soon as it 
discovers that the lists are not identical. 


4.11.2.3 List Depth 


Another handy operation on nonrecursive lists is the function that computes the 
depth of a list. The depth of the empty list is defined to be zero and, in general, 


_ |0 ifs isan atom 
depths) = 1) 4 max {depth(x,), «+, depth (x,)} it's is the list (ty, <*-s%9).0 21 
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Function Depth (Program 4.37) is a very close transformation of the 
definition, which is itself recursive. 


4 Driver 

template <class T> 

int GenList <T >::Depth() 

{// Compute the depth of a non-recursive list. 


return Depth (first); 
} 
4 Workhorse 
template <class T> 
int GenList <T >::Depth(GenListNode <T > *s) 
{ 
if ('s) return 0; // empty list 
GenListNode <T > *current = 3 
int m = 0; 
while (current) { 
if (current Stag) m= max (m, Depth (current >down)); 
current = current next; 
} 
return in +1; 
} 


Program 4.37: Computing the depth of a list 


4.11.3 Reference Counts, Shared and Recursive Lists 


In this section we shall consider some of the problems that arise when lists are 
allowed to be shared by other lists and when recursive lists are permitted. Shar- 
ing of sublists can, in some situations, result in great savings in storage used, as 
identical sublists occupy the same space. To facilitate specifying shared sublists, 
we extend the definition of a list to allow for naming of sublists. A sublist 
appearing within a list definition may be named through the use of a list name 
preceding it. For example, in the list A = (a, (b,c), the sublist (b,c) could be 
assigned the name Z by writing A = (@,Z(b,c)). In fact, to be consistent, we 
would then write A (a,Z{b.c)) which would define the list A as above. 

Lists that are shared by other lists, such as list B of Figure 4.32, create 
problems when you wish to add or delete a node at the front. If the first node of 
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B is deleted, it is necessary to change the pointers from list C to point to the 
second node. If a new node is added, pointers from C have to be changed to 
point to the new first node. However, we normally do not know all the points 
from which a particular list is being referenced. (Even if you did have this infor- 
mation, addition and deletion of nodes could require a large amount of time.) 
This problem is easily solved through the use of header nodes. If you expect to 
perform any additions and deletions at the front of lists, then the use of a header 
node with each list or named sublist will eliminate the need to retain a list of alt 
pointers to any specific list. If each list is to have a header node, then lists (1) to 
(4) are represented as in Figure 4.35. The values in the data/down fields of the 
header nodes is the reference count (defined below) of the corresponding list. 
Even in situations in which you do not wish to add or delete nodes from lists 
dynamically, as in the case of multivariate polynomials, header nodes prove use- 
ful in determining when the nodes of a particular structure may be retumed to the 
storage pool. 


tag data/down next 


Oe canes 
[tett] td = B=@@.0) 


bfirst 


first 


afirst —>Lt | 2] f(t] a [+e [si fd 


D=(a,D) 


Figure 4.35: Structure with header nodes for lists (1) to (4) 


Whenever lists are being shared by other lists, we need a mechanism to 
help determine whether or not the list nodes may be physically returned to the 
available-space list. This mechanism is generally provided through the use of a 
teference count maintained in the header node of each list. Since the data field 
of the header nodes is free, the reference count is maintained in this field. 
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(Altematively, a third variant may be introduced, with tag having three possible 
values: 0, 1, and 2.) The reference count of a list is the number of pointers 


{either program variables or pointers from other lists) to that list. The reference 
counts for the lists are 


(1) afirst ref = 1 accessible only via a.first 


(2) _ b.first ref = 3 pointed to by byfirst and two pointers from c 
Q) _ c.first ref = 1 accessible only via c,first 
(4) d.first ref = 2 accessible via d.first and one pointer from itself. 
Now a call to ¢"GenList <T>() (list destructor) should result only in a 
decrementing by 1 of the reference counter of ¢.first. Only when the reference 
count becomes zero are the nodes of t deleted. The same is to be done with the 
sublists of 1, 
Suppose we change the definition of GenListNode<T> to 
template <class T> 
class GenListNode <T> 
{ 
friend class GenList <T >; 
private: 
GenListNode <T > next; 
int tag; // 0 for data, | for down, 2 for ref 
union { 
T data; 
GenListNode <T > *down; 
int ref; 
hs 
h 


A recursive function to delete a list is given in Program 4.38. The workhorse 
proceeds by examining the top-level nodes of a list whose reference count has 
become zero, Any sublists encountered are deleted recursively, and finally, the 
top-level nodes are linked into the available-space list. 


A call to b-GenList <T >(), where b is list (ii) of Figure 4.35, now has only 


the effect of decreasing the reference count of 6 to 2. Such a call followed by a 
call toc. GenList <T >() results in 


(1) the reference count of ¢ becomes zero 


(2) b-first ref becomes | when the second top-level node of c is processed 
(3) b-firstref becomes 0 when the third top-level node of ¢ is processed; 
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4 Driver 

template <class T> 

GenList <T >: GenList() 

{// Each header node has a reference count. 


if (first) 
{ 
Delete (first); 
first = 0; 
} 
) 
4 Workhorse 
void GenList <T >::Delete(GenListNode <T>* x) 
{ 
xref ——; If decrement reference count of header node. 
if (1x ref) 
{ 
GenListNode <T > *y = x; /! y traverses top level of x. 
while (y next) { y = y next; if (y Stag = 1) Delete (y adown);} 
y~next = av; // attach top-level nodes to av list 
av=x3 
} 
} 


Program 4.38: Deleting a list recursively 


now, the five nodes of list B(a, (b,c)) are retumed to the available-space 
list 


(4) the top-level nodes of c are linked into the available-space list. 


The use of header nodes with reference counts solves the problem of determining 
when nodes are to be physically freed in the case of shared sublists. However, 
for recursive lists, the reference count never becomes zero. d.“GenList <T >() 
results in d. first ref becoming one. The reference count does not become zero, 
even though this list is no longer accessible either through program variables or 
through other structures. The same is true in the case of indirect recursion (Fig- 
ure 4.36). After calls to r.GenList <T>(Q and s.-GenList <T>(), r.firstref= | 
and s.first—>ref= 2 but the structure consisting of r and s is no longer accessible. 
So, its nodes should have been returned to the available-space list. 
Unfortunately, there is no simple way to supplement the list structure of 
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To] 


rfirst —A 7727 4-71] Tt] 
——— 


vie Toh 


r=A(B,B) and s = B(A) 


Figure 4.36: Indirect recursion of lists r= A ands =A 


Figure 4.36 so as to be able to determine when recursive lists may be physically 
deleted. It is no longer possible to return all free nodes to the available-space list 
when they become free. When recursive lists are being used, it is possible to run 
out of available space, even though not all nodes are in use. 


EXERCISES 


1 


Describe how a multivariate polynomial may be represented using the fol- 
lowing node structure that can be used to represent any generalized list: 


tag =false/true | data/down [_next | 


Write a nonrecursive version of GenList <T>:GenList <T>() (Program 
4,38). 


Write a nonrecursive version of operator== (Program 4.36). 
Write a nonrecursive version of GenList <T >::Depth (Program 4.37). 


Write a function that inverts an arbitrary nonrecursive list / with no shared 


sublists, The sublists of ! also are inverted. For example, if f= (a.(b,¢)), 
then inverse (1) = ((c,b),a). 


Devise a function that produces the list representation of an arbitrary list, 


given its linear form as a string of atoms, commas, blanks, and parentheses. 


For example, for the input ! = (a, (b,c)), your function should produce the 
structure of Figure 4.37. 
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tag data next 


Lfirst 


Figure 4.37: Structure for Exercise 6 


7. One way to represent generalized lists is use nodes that have two fields 
plus a table that contains all atoms and list names, together with pointers to 
these lists. Let the two fields of each node be named alink and blink. Then 
blink either points to the next node on the same level, if there is one, or it is 
0. The alink points either to a node at a lower level or, in the case of an 
atom or list name, to the appropriate entry in the symbol table. For exam- 
ple, the list B(A, (D,E),( ),B) would have the representation given in Fig- 
ure 4.38. The notation d* means a pointer to the first node of list D. 


name type address 


symbol table 


Figure 4.38: Representation for Exercise 7 


(The list names D and E were already in the table at the time the list B was 
input. A was not in the table and is assumed to be an atom.) 
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The symbol table retains a type bit for each entry. This bit is 1 if the entry 
is a list name and 0 for an atom. The NIL atom may be in the table, or 
alink can be set to 0 to represent the NIL atom. Write the C++ function 
operator>>, which inputs a list in parenthesis notation and sets up its 
linked representation as above, Note that no header nodes are in use. The 
following functions may be used by operator>>: 


(a) int Get(a) searches the symbol table for the name a. ~1 is returned if 
is not found in the table; otherwise, the position of a in the table is 
retumed. 


(b) Put(n,t,q) enters the tiple (n,t,a) into the table. If the name n is 
already in the table, then the type and address fields of the old entry 
are updated to ¢ and a, respectively. 


(c) NextToken() gets the next token in the input list. (A token may be a 
list name, atom, ‘(’,‘)' or‘, A ‘# is retumed if there are no more 
tokens.) 


Write code for all functions used by you. You may assume that the input 
list is syntactically correct. If a sublist is labeled, as in the list 
C(D,E(F,G)), the structure should be set up as in the case C(D,(F,G)), 
and E should be entered into the symbol table as a list with the appropriate 
starting address. 


8, What goes wrong when Program 4.35 is used to copy lists that have shared 
sublists? 


9, Write and test a C++ template class for generalized lists with reference 
counts. You must include a copy constructor and destructor for your class 
as well as functions to determine the depth of a list and to determine 
whether two lists are identical. What is the time complexity of your func- 
tions? 


CHAPTER 5 


Trees 


5.1 INTRODUCTION 
5.1.1 Terminology 


In this chapter we shall study a very important data object, the tree. Intuitively, a 
tree structure means that the data are organized in a hierarchical manner. One 
very common place where such a structure arises is in the investigation of 
genealogies. There are two types of genealogical charts that are used to present 
such data: the pedigree and the lineal chart. Figure 5.1 gives an example of 
each. 5S 

The pedigree chart of Figure 5.1(a) shows someone's ancestors, in this case 
those of Dusty, whose two parents are Honey Bear and Brandy. Brandy's 
parents are Coyote and Nugget, who are Dusty’s grandparents on her father's 
side. The chart continues one more generation back to the great-grandparents. 
By the nature of things, we know that the pedigree chart is normally two-way 
branching, though this does not allow for inbreeding. When inbreeding occurs, 
we no longer have a tree structure unless we insist that each occurrence of 
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Honey Bear Brandy 


————— 
Brumhilde Terry Coyote Nugget 


[ a 


Gill Tansey Tweed Zoe ~—«Crocus. Primrose. © Nous Belle 
(a) Pedigree 


Proto Indo-European 


Italic Hellenic Germanic 


== 


Same meray 
Osco-Umbrian Latin Greek North West 


Oscan Umbrian SpanishFrench Uolian IcelandiNorwegianSwedish Low HighYiddish 
(b) Lineal 


Figure 5.1: Two types of genealogical charts 


breeding is separately listed. Inbreeding may occur frequently when describing 
family histories of flowers or animals. , 

The lineal chart of Figure 5.1(b), though it has nothing to do with people, is 
still a genealogy. It describes, in somewhat abbreviated form, the ancestry of the 
modern European languages. Thus, this is a chart of descendants rather than 
ancestors, and each item can produce several others. Latin, for instance, is the 
forebear of Spanish, French, and Italian. Proto Indo-European is a prehistoric 
language presumed to have existed in the fifth millenium B.c, This tree does not 
have the regular structure of the pedigree chart, but it is a tree structure neverthe- 
less. 

With these two examples as motivation, let us define formally what we 
imean by a tree. 


Definition: A tree is a finite set of one or more nodes such that 
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(Lt) There is a specially designated node called the root. 


(2) The remaining nodes are partitioned into n >0 disjoint sets T), --+. Ty. 
where each of these sets is a tree. 7), ---, 7, are called the subtrees of 
the root. G 


Notice that this is a recursive definition. If we return to Figure 5.1, we see 
that the roots of the trees are Dusty and Proto Indo-European. Tree (a) has two 
subtrees, whose roots are Honey Bear and Brandy; tree (b) has three subtrees, 
with roots Italic, Hellenic, and Germanic. The condition that T;, ---, T, be dis- 
joint sets prohibits subtrees from ever connecting together (i.c., there is no 
cross-breeding). It follows that every item in a tree is the root of some subtree of 
the whole. For instance, Osco-Umbrian is the root of a subtree of Italic, which 
itself has two subtrees with the roots Oscan and Umbrian. Umbrian is the root 
of a tree with no subtrees. 

There are many terms that are often used when referring to wees. A node 
stands for the item of information plus the branches to other nodes. Consider the 
tree in Figure 5.2. This tree has 13 nodes, each item of data being a single letter 
for convenience. The root is A, and we will normally draw trees with the root at 
the top. 


LEVEL 


Figure 5.2: A sample tree 


The number of subtrees of a node is called its degree. The degree of A is 3, 
of C is 1, and of F is zero. Nodes that have degree zero are called leaf or termi- 
nal nodes. {K,L,F,G.M,1,J} is the set of leaf nodes. Consequently, the other 
nodes are referred to as nonterminals. The roots of the subtrees of a node X are 
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the children of X. X is the parent of its children, Thus, the children of D are H, 
J, and J; the parent of D is A. Children of the same parent are said to be siblings. 
H, 1, and J are siblings. We can extend this terminology if we need to so that we 
can ask for the grandparent of M, which is D, and so on. The degree of a tree is 
the maximum of the degree of the nodes in the tree. The tree of Figure 5.2 has 
degree 3. The ancestors of a node are all the nodes along the path from the root 
to that node. The ancestors of M are A, D, and H. 

The level of a node is defined by letting the root be at level one*. If a node 
is at level /, then its children are at level / + 1. Figure 5.2 shows the levels of all 
nodes in that tree. The height or depth of a tree is defined to be the maximum 
Jevel of any node in the tree. Thus, the depth of the tree in Figure 5.2 is 4. 


5.1.2 Representation of Trees 


5.1.2.1 List Representation 


There are several ways to draw a tree besides the one presented in Figure 5.2. 
One useful way is as a list. The tree of Figure 5.2 could be written as the list 


(A(B(E(K,L),F),C(G),D(H(M),1,J))) 


The information in the root node comes first, followed by a list of the subtrees of 
that node. This way of drawing trees leads to a memory representation of trees 
that is the same as that used for generalized lists in Chapter 4. Figure 5.3 shows 
the resulting memory representation for the tree of Figure 5.2. If we use this 
representation, we can make use of many of the general functions that we origi- 
nally wrote for handling lists. 

For several applications it is desirable to have a representation that is spe- 
cialized to trees. One possibility is to represent each tree node by a memory 
node that has fields for the data and pointers to the tree node’s children. Since 
the degree of each tree node may be different, we may be tempted to use memory 
nodes with a varying number of pointer fields. However, as it is often easier to 
write algorithms for a data representation when the node size is fixed, in practice 
‘one uses only nodes of a fixed size to represent tree nodes. For a tree of degree 
k, we could use the node structure of Figure 5.4. Each child field is used to point 


to a subtree. Lemma 5.1 shows that using this node structure is very wasteful of 
space, 


“Note that some authors define the level ofthe root to be O. Lenin Son 
feng $v AY 
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Lio 


tag fields not shown 


Figure 5,3: List representation of the tree of Figure 5.2 


DATA | CHILD 1 CHILD 2 | -- | CHILDk 


Figure 5.4: Possible node structure for a tree of degree k 


Lemma 5.1: If Tis a k-ary tree (ie., a tree of degree k) with n nodes, each hav- 
ing a fixed size as in Figure 5.4, then n(k — 1) + 1 of the nk child fields are 0, 
n2i. 


Proof: Since each non-zero child field points to a node and there is exactly one 
pointer to each node other than the root, the number of non-zero child fields in an 
n-node tree is exactly n — 1. The total number of child fields in a k-ary tree with 
n nodes is nk. Hence, the number of zero fields is nk —(n —1)=n(k-1) +1. 
a 


We shall develop two specialized fixed-node-size representations for trees. 
Both of these require exactly two link, or pointer, fields per node. 
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5.1.2.2 Left Child-Right Sibling Representation 


Figure 5.5 shows the node structure used in the left child—right sibling represen- 
tation. 


data 
left child right sibling, 


Figure 5.5: Left child-right sibling node structure 


To convert the tree of Figure 5.2 into this representation, we first note that 
every node has at most one leftmost child and at most one closest right sibling. 
For example, in Figure 5.2, the leftmost child of A is B, and the leftmost child of 
Dis H. The closest right sibling of B is C, and the closest right sibling of H is /. 
Strictly speaking, since the order of children in a tree is not important, any of the 
children of a node could be the leftmost child, and any of its siblings could be the 
closest right sibling. For the sake of definiteness, we choose the nodes based on 
how the tree is drawn. The /eft child field of each node points to its leftmost 
child (if any), and the right sibling field points to its closest right sibling (if any). 


Figure 5.6 shows the tree of Figure 5.2 redrawn using the left child-right sibling 
representation. 


Figure 5.6: Left child-night sibling representation of tree of Figure 5.2 
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5.1.2.3. Representation as a Degree-Two Tree 


To obtain the degree-two tree representation of a tree, we simply rotate the 
right-sibling pointers in a left child-right sibling tree clockwise by 45 degrees. 
This gives us the degree-two tree displayed in Figure 5.7. In the degree-two 
Tepresentation, we refer to the two children of a node as the left and right chil- 
dren. Notice that the right child of the root node of the tree is empty. This is 
always the case since the root of the tree we are transforming can never have a 
sibling. Figure 5.8 shows two additional examples of trees represented as left 
child-right sibling trees and as left child-right child (or degree-two) trees. Left 
child-right child trees are also known as binary trees. 


Figure 5.7: Left child-right child tree representation of tree of Figure 5.2 


EXERCISES 


1. Write a function to input a tree given as a generalized list (€.g., 
(A (B(E(K,L),F),C(G),D(H(M),/,J)))) and create its internal representa- 
tion using nodes with three fields: rag, data, and link. 
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Figure 5.8: Tree representations 


2. Write a function that reverses the process in Exercise 1 and takes a pointer 
to a tree and outputs it as a generalized list. 


3 


[Programming Project] Write the C++ class definition for trees using the 


list representation described in this section. Write the following C++ func- 


tions. 


(a) 


{b) 
{ce} 


{d) 
te) 


[operator>>()}: accept a tree represented as a parenthesized list as 


input and create the generalized list representation of the tree (see 
Figure 5.3) 


(copy constructor]: initialize a tree with another tree represented as a 
generalized list 


{operator==()}: test for equality between two trees represented as 
generalized lists 


[destructor]: delete a tree represented as a generalized list 
[operator<<()): output a tree in its parenthesized list notation 


Test the correctness of your functions using suitable test data. 
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5.2 BINARY TREES 


5.2.1 The Abstract Data Type 


We have seen that we can represent any tee as a binary tree. In fact, binary trees 
are an important type of tree structure that occurs very often. Binary trees are 
characterized by the fact that any node can have at most two branches (i.e., there 
is no node with degree greater than two). For binary trees we distinguish 
between the subtree on the left and that on the right, whereas for trees the order 
of the subtrees is irrelevant. Also, a binary tree may have zero nodes. Thus, a 
binary wee is really a different object from a tree. 


Definition: A binary tree is a finite set of nodes that either is empty or consists 
of a root and two disjoint binary trees called the left subtree and the right sub- 
tree.0 


ADT 5.1 contains the specification for the binary tree data structure. This 
specification defines only a minimal set of operations on binary trees, which we 
use as a foundation on which to build additional operations. 

Let us carefully review the distinctions between a binary tree and a tree. 
First, there is no tree having zero nodes, but there is an empty binary tee. 
Second, in a binary tree we distinguish between the order of the children; in a 
tree we do not. Thus, the two binary trees of Figure 5.9 are different, since the 
first binary tree has an empty right subtree, while the second has an empty left 
subtree. Viewed as trees, however, they are the same, despite the fact that they 
are drawn slightly differently. 

Figure 5.10 shows two special kinds of binary trees. The first is a skewed 
tree, skewed to the left, and there is a corresponding tree that skews to the right. 
The tree of Figure 5.10(b) is called a complete binary tree. This kind of binary 
tree will be defined formally later. Notice that all leaf nodes are on adjacent lev- 
els. The terms that we introduced for trees such as degree, level, height, leaf, 
parent, and child all apply to binary trees in the natural way. 


5.2.2 Properties of Binary Trees 


Before examining data representations for binary trees, let us make some obser- 
vations about such trees. In particular, we want to determine the maximum 
number of nodes in a binary tree of depth & and the relationship between the 
number of leaf nodes and the number of degree-two nodes in a binary tree. 
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template <class 7> 
class BinaryTree 
{# objects: A finite set of nodes either empty or consisting of a 


H root node, left BinaryTree and right BinaryTree. 
public: 


BinaryTree (); 
# creates an empty binary tree 


bool /sEmpty (); 
/ return true iff the binary tree is empty 


BinaryTree(BinaryTree <T>& bt 1, T& item, BinaryTree <T >& bt 2); 
#f creates a binary tree whose left subtree is bt 1, whose right subtree 
JH is bt2, and whose root node contains item 


BinaryTree <T > LefiSubtree (); 
H return the left subtree of *this 


BinaryTree <T > RightSubtree (); 
i return the right subtree of *this 


T RootData(); 
df return the data in the root node of *this 


ADT 5.1: Abstract data type BinaryTree 


Figure 5.9: Two different binary tees 
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LEVEL 


w 


Figure 5.10: Skewed and complete binary trees 


Lemma 5.2 [Maximum number of nodes): 

(1) The maximum number of nodes on level i of a binary tree is 2'~', 72 1. 
(2) The maximum number of nodes in a binary tree of depth k is 2 -1k21. 
Proof: 


(1) The proof is by induction on i. 


Induction Base: The root is the only’ node on level i= 1. Hence, the maximum 
number of nodes on level i= 1 is 2°"! = 2° = 1, 


Induction Hypothesis: Let i be an arbitrary positive integer greater than 1. 
Assume that the maximum number of nodes on level i- | is 2°? 


Induction Step: The maximum number of nodes on level i - J is 7 st by the 
induction hypothesis. Since each node in a binary tree has a maximum degree of 
2, the maximum number of nodes on level i is two times the maximum number of 
nodes on level i-1, or 2/7. 


254 Trees 


(2) The maximum number of nodes in a binary tree of depth k is 


k k 
© (maximum number of nodes on level ) = $2!" = 24-19 
11 ist 

Lemma 5.3 [Relation between number of leaf nodes and degree-2 nodes): For 


any nonempty binary tree, 7, if no is the number of leaf nodes and m2 the number 
of nodes of degree 2, then ng =n2 +1. 


Proof: Let n, be the number of nodes of degree one and x the total number of 
nodes. Since all nodes in 7 are at most of degree two, we have 


nN=ngtny) tng (S.1) 


Tf we count the number of branches in a binary tree, we see that every node 
except the root has a branch leading into it. If B is the number of branches, then 


n= B+]. All branches stem from a node of degree one or two, Thus, B =n, + 
2n2. Hence, we obtain 


n=B+1l=n,)+2n24+1 (5.2) 
Subtracting Eq. (5.2) from Eq. (5.1) and rearranging terms, we get 
mge=nz+10 


In Figure 5.10(a). 29 = 1 and n4 = 0; in Figure 5.10(b), no = 5 and ny =4. 
‘We are now ready to define full and complete binary trees. 


Definition: A full binary tree of depth k is a binary tree of depth k having 2-4 
nodes, k 20. 0 


By Lemma 5.2, 24} is the maximum number of nodes in a binary tree of 
depth k. Figure 5.11 shows a full binary tree of depth 4. Suppose we number the 
nodes in a full binary tree starting with the root on level 1, continuing with the 
nades on level 2, and so on. Nodes on any level are numbered from left to right. 
This numbering scheme gives us the definition of a complete binary tree. 


Definition: A binary tree with n nodes and depth k is complete iff its nodes 
correspond to the nodes numbered from | to n in the full binary tree of depth k.O 


From Lemma 5.2, it follows that the height of a complete binary tree with n 
nodes is []logo(2 + 1}]. (Note that [x] is the smallest integer 2x.) 
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Figure 5.11: Full binary tree of depth 4 with sequential node numbers 


5.2.3 Binary Tree Representations 


5.2.3.1 Array Representation 


The numbering scheme used in Figure 5.11 suggests our first representation of a 
binary tree in memory. Since the nodes are numbered from | to n, we can use a 
one-dimensional array to store the nodes. (If the C++ array is used to represent 
the tree, the zero’th position is left empty.) Using Lemma 5.4 we can easily 
determine the locations of the parent, left child, and right child of any node, i, in 
the binary tree. 


Lemma 5.4: If a complete binary tree with n nodes is represented sequentially, 
then for any node with index i, 1 <i $n, we have 


(1) parent (i) is at | i/2 | ifi# 1. If i= 1, iis at the root and has no parent. 
(2) leftChild (i) is at 2i if 2i <n. If 2i>n, then i has no left child. 


(3) rightChild (i) is at 2) + 1 if 28+ 1<_n. If 2i+1> 2, then é has no right 
child. 


Proof: We prove (2). (3) is an immediate consequence of (2) and the numbering 
of nodes on the same level from left to right. (1) follows from (2) and (3). We 
prove (2) by induction oni. For i= 1, clearly the left child is at 2 unless 2 > n, in 
which case i has no left child. Now assume that for all j, 1 <j $i, leftChild (/) is 
at 2j. Then the two nodes immediately preceding lefiChild (i +1) are the right 
and left children of i. The left child is at 2i. Hence, the left child of i + 1 is at 2i 
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+25 2(i + 1) unless 2(¢ + 1) >a, in which case + 1 has no left child. 


This representation can clearly be used for all binary trees, though in most 
cases there will be a lot of unutilized space. Figure 5.12 shows the array 
representation for both trees of Figure 5.10. For complete binary trees such as 
the one in Figure 5.10(b), the representation is ideal, as no space is wasted. For 
the skewed tree of Figure 5.10(a), however, less than half the array is utilized. In 
the worst case a skewed tree of depth k will require 2‘—1 spaces, Of these, only 
k will be used. 


tree 
mM = 
a 
i B 
(3 
(4) [> | 
15] | = | 
(6) F 
7 G 
(8) | 4 | 
0) LT] 
; (b) Tree of Figure 5.10(b) 

(16) 


(a) Tree of Figure 5.10(a) 


Figure 5.12: Array representation of the binary trees of Figure 5.10 


5.2.3.2 Linked Representation 


Although the array representation is good for complete binary trees, it is wasteful 
for many other binary trees. In addition, the representation suffers from the gen- 
eral inadequacies of sequential sepresentations. Insertion and deletion of nodes 
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from the middle of a tree require the movement of potentially many nodes to 
reflect the change in level number of these nodes. These problems can be over- 
come easily through the use of a linked representation. Each node has three 
fields, leftChild, data, and rightChild. As with linked lists, we will use two 
classes to define a tree using a linked representation. 
template <class T> class Tree; // forward declaration 


template <class 7> 
class TreeNode { 
friend class Tree <T >; 
private: 
T data; 
TreeNode <T > *lefiChild; 
; TreeNode <T > *rightChild; 
3 
template <class 7> 
class Tree { 
public: 
Hf Tree operations 


private: 
TreeNode <T> *root; 


3 
We shall draw a tree node using either of the representations of Figure 5.13. 


| leftChita data | rightChild (sa 


lefiChild rightChild 


Figure 5.13: Node representations 


Although with this node structure it is difficult to determine the parent of a 
node, we shall see that for most applications, this node structure is adequate. If it 
is necessary to be able to determine the parent of random nodes, then a fourth 
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field, parent, may be included in the class TreeNode. The representation of the 
binary trees of Figure 5.10 using this node structure is given in Figure 5.14. The 
root of the tree is stored in the data member root of Tree. This data member 
serves as the access pointer to the tree, 


Figure 5.14; Linked representation for the binary trees of Figure 5.10 


EXERCISES 


1. For the binary tree of Figure 5.15, list the leaf nodes, the nonleaf nodes, and 
the level of each node. 

2. What is the maximum number of nodes in a k-ary tree of height h? Prove 
your answer. 

3. Draw the internal memory representation of the binary tree of Figure 5.15 
using (a) sequential and (b) linked representations. 

4. Extend the array representation of a complete binary tree to the case of 
complete trees whose degree is d, d>1. Develop formulas for the parent 
and children of the node stored in position i of the artay. 
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Figure 5.15: Binary tree for Exercise 1 


5.3. BINARY TREE TRAVERSAL AND TREE ITERATORS 


5.3.1 Introduction 


There are many operations that we often want to perform on trees. One notion 
that arises frequently is the idea of uaversing a tree or visiting each node in the 
tree exactly once. When a node is visited, some operation (such as outputting its 
data field) is performed on it. A full traversal produces a linear order for the 
nodes in a tree. This linear order, given by the order in which the nodes are 
visited, may be familiar and useful. When traversing a binary tree, we want to 
treat each node and its subtrees in the same fashion. If we let L, V, and R stand 
for moving left, visiting the node, and moving right when at a node, then there 
are six possible combinations of traversal: LVR, LRV, VLR, VRL, RVL, and RLV. 
If we adopt the convention that we traverse left before right, then only three 
traversals remain: LVR, LRV, and VLR. To these we assign the names inorder, 
postorder, and preorder, respectively, because of the position of the V with 
respect to the L and the R. For example, in postorder, we visit a node after we 
have traversed its left and right subtrees, whereas in preorder the visiting is done 
before the traversal of these subtrees. 

There is a natural correspondence between these traversals and producing 
the infix, postfix, and prefix forms of an expression. Consider the binary wee of 
Figure 5.16. This tree contains an arithmetic expression with the binary opera- 
tors add (+), multiply (*), and divide (/) and the variables A, B, C, D, and E. For 
each node that contains an operator, its left subtree gives the left operand and its 
right subtree the right operand. We use this tree to illustrate each of the traver- 
sals. 
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Figure 5.16; Binary tree with arithmetic expression 


§.3.2 Inorder Traversal 


Informally, inorder traversal calls for moving down the tee toward the left until 
you can go no farther. Then you “‘visit’’ the node, move one node to the right 
and continue. If you cannot move to the right, go back one more node. A pre- 
cise way of describing this traversal is by using recursion as in Program 5.1. 

Recursion is an elegant device for describing this traversal. Figure 5.17 is 
a trace of how function Inorder(TreeNode<T> +) (Program 5.1) works on the 
tree of Figure 5.16, Read down the left column first and then the right one. Fig- 
ure 5.17 assumes that the Visit function has a single line of code: 


cout << currentNode data; 


The elements get output in the following order: 
A/B*C*D+E 


which is the infix form of the expression. 


5.3.3 Preorder Traversal 


The C++ vode for the second form of traversal, preorder, is given in Program 
5.2. In words, we would say ‘‘visit a node, traverse left, and continue. When 
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1 template <class 7> 

2 void Tree <T>::Inorder () 

3 {/ Driver calls workhorse for traversal of entire tree. The driver is 
4 if declared as a public member function of Tree. 

5 Inorder(root); 

6} 


7 template <class T> 
8 void Tree <T >::inorder(TreeNode <T > *currentNode) 
9 {// Workhorse traverses the subtree rooted at currentNode. 
10 / The workhorse is declared as a private member function of Tree. 
11 if (currentNode) { 
12 Inorder(currentNode —leftChild); 
13 Visit (currentNode ); 
14 inorder (currentNode —rightChild); 


Program 5.1: Inorder traversal of a binary tee 


you cannot continue, move right and begin again or move back until you can 
move right and resume.’’ The nodes of Figure 5.16 would be output in preorder 
as 

+**/ABCDE 


which we recognize as the prefix form of the expression. 


5.3.4 Postorder Traversal 


The code for postorder traversal is given in Program 5.3. On the tree of Figure 
5.16, this function produces the following output: 


AB/C#D*E+ 


which is the postfix form of our expression. 
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Call of Value in Call of Value in 
inorder __currentNode Action inorder _currentNode Action 
Driver + 10 Cc. 
1 * ll 0 
2 * 10 Cc cout << 'C’ 
3 / 12 0 
4 A 1 * cout << "+" 
5 0 13 D 
4 A cout <<’A’ 14 0 
6 0 13 D cout << "D’ 
3 / cout << 7” 15 0 
7 B Driver + cout << "+" 
8 i) 16 E 
7 B cout << 'B’ 17 0 
9 0 16 E cout <<’E’ 
2 * cout <<"*" 18 0 


Figure 5,17: Trace of Program 5.1 
5.3.5 Iterative Inorder Traversal 


Since the tree is a container class, we would like to implement an iterator for the 
class Tree. Suppose that we have decided that our iterator will sequence through 
the elements in the tree in inorder. To implement such an inorder iterator, we 
first need to implement the inorder traversal method without using recursion. A 
nonrecursive code for inorder traversal is given in Program 5.4. This program 
USES-A stack as defined in ADT 3.1. 


Definition: We say that a data object of Type X USES-A data object of Type Y if 
a Type X object uses a Type Y object to perform a task. This relationship is typi- 
cally expressed by employing the Type ¥ object in a member function of Type X. 
The Type ¥ object may be passed as an argument to the member function or used 
as a local variable in the function. The USES-A relationship is similar to the 1S- 
IMPLEMENTED-IN-TERMS-OF relationship. The difference is that the degree 
to which the Type Y object is used is less in the USES-A relationship. 2 


For our purposes, the stack template of ADT 3.1 is instantiated to 
TreeNode<T>*. 


Analysis of NonrecInorder: Let n be the number of nodes in the tree. If we 
consider the action of Program 5.4, we note that every node of the tree is placed 
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I template <class 7> 
2 void Tree <T >::Preorder() 


3 {4 Driver. 
4 Preorder (root); 
3) 


6 template <class 7> 
7 void Tree <T >::Preorder(TreeNode <T > *currentNode) 
8 {// Workhorse. 
9 if (currentNode) { 
10 Visit (currentNode ); 
11 Preorder (currentNode —lefiChild); 
12 Preorder (currentNode —rightChild); 
} 


Program 5.2: Preorder traversal of a binary tree 


1 template <class T> 

2 void Tree <T >::Postorder() 
3 {4 Driver. 

4 Postorder (root); 

5 


6 template <class 7> 
7 void Tree <T >::Postorder(TreeNode <T > *currentNode) 
8 {/ Workhorse. 
9 if (currentNode) { 
10 Postorder (currentNode —lefiChild); 
ll Postorder(currentNode >rightChild); 
12 Visit (currentNode ); 


Program 5.3: Postorder traversal of a binary tree 
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1 template <class T> 

2 void Tree <T >::Nonrecinorder() 

3 {// Nonrecursive inorder traversal using a stack. 

4 Stack <TreeNode <T>*> s; /f declare and initialize stack 
5 TreeNode <T > *currentNode = root; 

6 while(1) { 


7 while (currentNode) { /! move down leftChild fields 

8 s.Push (currentNode); // add to stack 

9 currentNode = currentNode leftChild; 
10} 


MW if (s.JsEmpry ()) return; 

12 currentNode = s.Top(); 

13 s.Pop(); Hf delete from stack 
14 Visit (currentNode ); 


15 currentNode = currentNode —>rightChild; 
16 


} 
17} 


Program 5.4: Nonrecursive inorder traversal 


on the stack once. Thus, the statements on lines 8, 9 and 11 to 15 are executed 2 


times. Moreover, currentNode will equal 0 once for every 0 link in the tree, 
which is exactly 


ng tn, =Ngotny tngtlenel 


Every step will be executed no more than some constant times n, so the time 
complexity is O(n). The run time can be reduced by a constant factor by elim- 
inating some of the unnecessary stacking (see Exercises). The space required 
for the stack is equal to the depth of the tree. This is at most a. 0 


We now use the function Nonrecinorder to obtain an inorder iterator for a 
tree. Rather than develop an iterator as intricate as a C++ forward iterator, we 
consider a simple iterator that has only a Next function, which returns a pointer 
to the next element in inorder. The key observation required to develop our 
simplified iterator is that each iteration of the while loop of lines 6-16 in Program 
5.4 yields the next element in the inorder traversal of the tree. We begin by 
defining the class Jnorderlterator, which is a nested class (and a friend) of Tree. 
The data members required for this class are the the two objects 5 and 
currentNode as used in Program 5.4. Program 5.5 contains the class definition of 
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Inorderlterator. The constructor for this class initializes currentNode to the tree 
root. The code implementing Next() is obtained by extracting lines 7-15 of Pro- 
gram 5.4 corresponding to a single iteration of the while loop. Instead of visiting 
the next element, we return this element. Program 5.6 gives the resulting code. 


class inorderlterator { 

public: 
Inorderlterator({currentNode = root;}; 
T * Next ()3 

Private 
Stack <TreeNode <T >*> s3 
TreeNode<T> *currentNode; 

k 


Program 5.5: Definition of a simple inorder iterator class 


T *Inorderlterator ::Next () 


while (currentNode) { 
3.Push (currentNode ); 
currentNode = currentNode —leftChild; 


} 

if (s.JsEmpty ()) return 0; g 
currentNode = s.Top(); 

s.Pop()3 

T& temp = currentNode data; 

currentNode = currentNode —rightChild; 

return &temp; 


} 


Program 5.6: Code for obtaining the next inorder element 


5.3.6 Level-Order Traversal 


Whether written iteratively or recursively, the inorder, preorder, and postorder 
traversals all require a stack. We now tum to a traversal that requires a queue. 
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This traversal, called level-order traversal, visits the nodes using the ordering 
suggested by the node numbering scheme of Figure 5.11. Thus, we visit the root 
first, then the root's left child, followed by the root’s right child. We continue in 
this manner, visiting the nodes at each new level from the leftmost node to the 
tightmost node. 


The code for this traversal, Program 5.7, uses the template class Queue of 
ADT 3.2. 


template <class 7> 
void Tree <T >::LevelOrder () 
{// Traverse the binary tree in level order. 
Queue <TreeNode <T>*> q; 
TreeNode <T > *currentNode = root; 
while (currentNode) { 
Visit (currentNode ); 
if (currentNode ~»leftChild) g.Push (currentNode —leftChild ); 
if (currentNode —rrightChild) q.Push (currentNode —rightChild); 
if (g.IsEmpty ()) return; 
currentNode = q.Front(); 
4-Pop ()s 


} 


Program 5.7: Level-order.traversal of a binary tree 


We begin by visiting the root and adding its children to the queue. The 
next node to visit is obtained from the front of the queue. Since a node’s children 
are at the next lower level, and we add the left child before the right child, the 
nodes are output using the ordering scheme found in Figure 5.11. The level- 
order traversal of the tree in Figure 5.16 is 


+*E*DICAB 


5.3.7 Traversal without a Stack 


Before we leave the topic of tree traversal, we shall consider one final question. 
1s binary tree traversal possible without the use of extra space for a stack? (Note 
that a recursive tree traversal algorithm also implicitly uses a stack.) One simple 
solution is to add a parent field to each node. Then we can trace our way back 
up to any root and down again. Another solution, which requires two bits per 
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node, represents binary trees as threaded binary wees. We study this in Section 
5.5. If the allocation of this extra space is too costly, then we can use the 
leftChild and rightChild fields to maintain the paths back to the root. The stack 
of addresses is stored in the leaf nodes. The exercises examine this algorithm 
more closely. 


EXERCISES 

1. Write out the inorder, preorder, postorder, and level-order traversals for the 
binary trees of Figure 5,10. 

2. Do Exercise | for the binary tree of Figure 5.11. 

3. Do Exercise 1 for the binary tree of Figure 5.15. 

4. Implement a forward iterator for Tree. Your iterator should waverse the tree 
in inorder, 

5. Implement a forward iterator for Tree. Your iterator should traverse the tree 
in Jevel order. 

6. Write a nonrecursive version of function Preorder (Program 5.2). 

7. Use the results of the previous exercise to implement a forward iterator, 
Preorderlterator, that traverse the tree in preorder. 

8. Write a nonrecursive version of function Postorder (Program 5.3). 


9. Use the results of the previous exercise to implement a forward iterator, 
Postiterator, that traverse the tree in postorder. 


10. {Programming Project]: Develop a complete C++ template class for 
binary trees. You must include a constructor, copy constructor, destructor, 
the four traversal methods of this section together with forward iterators for 
each. Include also the remaining functions specified in ADT 5.1. 

31. Rework Nonrecinorder (Program 5.4) so that it is as fast as possible. 
(Hint: Minimize the stacking and the testing within the loop.) 

12. Program 5.8 performs an inorder traversal without using a stack. Verify 
that the code is correct by running it on a variety of binary trees that cause 
every Statement to execute at least once. 


template <class T> 
void Tree <T >::NoStackinorder () 
{// Inorder traversal of binary tree using a fixed amount of additional storage. 
if (!root) return; // empty binary tree 
TreeNode <T > *top = 0, *lastRight =0, *p, *q, *r, #15 
Pq=root; 
while (1) { 
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while (1) { 
if (1p leftChild) && (!p rightChild)) { // leaf node 
Visit (p); breaks 


if (tp leftChild) { i! visit p and move to p—rightChild 
Visit (p)s 
r=p—rightChild; p srightChild = q; 
hal 23 de) 


else { // move to p leftChild 
r= p—leftChild; p leftChild = q3 
q=PspHars 


} 4 end of inner while 
4! pisa leaf node, move upward to a node whose 
Hf tight subtree has not yet been examined 
TreeNode <T > *av =p; 
while (1) { 
if (p == root) return; 
if (1g leftChild) { 1 q is linked via rightChild 
r=q-vrightChild; q—rightChild = p; 
PHhqes 


} 

else if (!q rightChild) { 1 q is linked via leftChild 
r=q-leftChild; qleftChild = p; 
P=Qqers Visit(p)s 


else // check if p is a rightChild of q 
if (g == lastRight) [ 
r= top; lastRight = rleftChild; 
top = r >rightChild; #/ unstack 
r—leftChild = r—rightChild = 0; 
r=q—rightChild; q —rightChild = p; 
PTBqay 


else { // pis leftChild of ¢ 
Visit (q)3 
av leftChild = lastRight; av >rightChild = top; 
top = av; lastRight = 43 
r=q—leftChild; q leftChild = p; i! restore link to p 
rl =q—rightChild; q >rightChild =r; 
perl 
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break; 


}4 end of inner while loop 
} // end of outer while loop 


Program 5.8: O(1) space inorder traversal 


13, Write a nonrecursive version of Postorder (Program 5.3) using only a fixed 
amount of additional space. (Use the ideas of the previous exercise.) 


14. Do the preceding exercise for the case of Preorder (Program 5.2). 


5.4 ADDITIONAL BINARY TREE OPERATIONS 


5.4.1. Copying Binary Trees 


Using the definition of a binary tree and the recursive version of the traversals, 
we can easily write other routines for working with binary trees. For instance, if 
we want to implement a copy constructor to initialize a binary tree with an exact 
copy of another binary tree, we can modify the postorder traversal algorithm 
only slightly to get Program 5.9, which assumes that TreeNode has a constructor 
that sets all three data members of a tree node. 


5.4.2 Testing Equality 


Another problem that is especially easy to solve using recursion is determining 
the equivalence of two binary trees. Binary trees are equivalent if they have the 
same topology and the information in corresponding nodes is identical. By the 
same topology we mean that every branch in one tree corresponds to a branch in 
the second in the same order and vice versa. The function operator==() calls 
the workhorse function Equal (Program 5.10), which traverses the binary trees in 
preorder, though any order could be used. 


5.4.3 The Satisfiability Problem 


Consider the set of formulas we can construct by taking variables x), x2, 43, 
-++, and the operators « (and), v (or), and ~ (not). These variables can hold 
only one of two possible values, true or false. The set of expressions that can be 
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template <class 7> 


Tree <T >::Tree(const Tree <T>& S) # driver 
{4 Copy constructor 


root = Copy (s.root); 


template <class T> 


TreeNode <T >* Tree <T>::Copy(TreeNode <T > *origNode) /f Workhorse 


Ww Retum a pointer to an exact copy of the binary tree rooted at origNode. 
if (lorigNode) return 0; 


return new TreeNode <T >(origNode data, 


Copy (origNode —>leftChild), 
} Copy (origNode —rightChild)); 


Program 5.9: Copying a binary tree 


template <class T> 
bool Tree <T>::operator==(const Tree& ¢) const 


{ 
} 


template <class 7> 
bool Tree <T >::Equal (TreeNode <T > *a, TreeNode <T > *b) 


return Equal (root,t.root); 


{// Workhorse. 
if ((!a) && (1b)) return true; // both a and b are 0 
return (a && b # both a and b are non-zero 


&& (a —sdata == b data) I data is the same 
&& Equal (a >leftChild,b leftChild) // left subtrees equal 


&& Equal (a -rightChild,b ~ rightChild)); 1 right subtrees equal 
} 


Program 5.10: Binary tee equivalence 
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formed using these variables and operators is defined by the following rules: 
(1) a variable is an expression 
(2) if x and y are expressions then x A y, x v y, and =x are expressions 


(3) parentheses can be used to alter the normal order of evaluation, which is 
not before and before or. 


This set defines the formulas of the propositional calculus (other operations such 
as implication can be expressed using A, v, and —). The expression 


XV GQA743) 


is a formula (read ‘‘x, or x2 and not x3"’). If x, and x3 are fatse and x2 is true, 
then the value of this expression is 


false v (true » = false) 
= false v true 
= true 


The satisfiabiliry problem for formulas of propositional calculus asks if there is 
an assignment of values to the variables that causes the value of the expression 
to be true. 

Again, let us assume that our formula is already in 2 binary tree, say 


(41 AnX2) Vv (ax) Ax3)V 903 


in the tree of Figure 5.18. The inorder traversal of this tree is 
X| AAX2V 4X1 AX3V 403 

which is the infix form of the expression. The most obvious algorithm to deter- 
mine satisfiability is to let (x1, x2, x3) take on all possible combinations of true 
and false values and to check the formula for each combination. For 1 variables 
there are 2” possible combinations of true = ¢ and false = f. For example, for 
n = 3, the eight combinations are: (1,4,1), (Qt), (SD, Qf.) itn. Gott), 
GLO. OS.F). The algorithm will take O(g 2"), or exponential time, where g is 
the time to substitute values for x,,x2, -- >, x, and evaluate the expression. 

To evaluate an expression, we traverse its tree in postorder. When visiting a 
node p, we compute the value of the expression represented by the subtree rooted 
at p. Recall that, in postorder, the left and right subtrees of a node are traversed 
before we visit that node. In other words, when we visit the node p, the subex- 
pressions represented by its left and right subtrees have been computed. So, 
when we reach the v node on level 2. the values of x) Ax and 4.x; 4x3 will 
already be available to us, and we can apply the rule for or. Notice that a node 
containing — has only a right branch, since — is a unary operator. 
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te ee 
Figure 5.18: Propositional formula in a binary tree 


For our satisfiability problem application, we shall instantiate the template 
class Tree with T = pair <Operator, bool>, where pair is the predefined C++ 
template structure with two data members first and second. The data types of 


these two data members are, respectively, Operator and bool. The data type 
Operator is defined by us as below. 


enum Operator {Not, And, Or, True, False}; 


Strictly speaking, True and False denote constants rather than operators. The 
first version of our algorithm for the satisfiability problem is Program 5.11. In 
this, 1 is the number of variables in the formula and formula is the binary tree 
that represents the formula. 

Program 5.12 provides the code for tasks to be performed when visiting a 
node of the expression tree. For simplicity, Program 5.12 assumes that for every 
leat node, its data. first field has been set either to True of False depending on the 
current truth assignment of the variable for that leaf node. 


EXERCISES 


1. Write a C++ function to count the number of leaf nodes in a binary tree. 
What is its computing time? 
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for each of the 2" possible truth value combinations for the a variables 
replace the variables by their values in the current truth value combination; 
evaluate the formula by traversing the tree it points to in postorder, 
if (formula. Data ().second ()) { cout << current combination; return;} 


cout << ‘‘no satisfiable combination’; 


Program 5.11: First version of satisfiability algorithm 


4 visit the node pointed at by p 
switch (p data. first) { 
case Not: p ~data.second = |p —srightChild data.second; break; 
case And: p >data.secon 
pleftChild —-data.second && p —srightChild data second; 
break; 
case Or: p ->data.second = 
p—lefiChild -»data.second |\ p >rightChild data. second; 
break; 
case True: p—>data.second = true; break; 
case False: p ->data.second = false; 


} 


Program 5.12: Visiting a node in an expression tree 


2. Write a C++ function, SwapTree (), that swaps the left and right children of 
every node of a binary tree. An example is given in Figure 5.19. 

3. Devise an external representation for the formulas in propositional cal- 
culus. Write a function that reads such a formula and creates its binary tree 
representation. What is the complexity of your function? 

4, {Destructor] Write a recursive function to delete all nodes in a binary tree. 
What is the complexity of your function? 

5. Write a recursive function to assign one tree to another [operator=]. What 
is the complexity of your function? 
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t.SwapTree () 


Figure 5.19: A SwapTree cxample 
5.5 THREADED BINARY TREES 
5.5.1 Threads 


If we look carefully at the linked representation of any binary tree, we notice that 
there are more 0-links than actual pointers. As we saw before, there are n + 1 0- 
links and 2n total links. A clever way to make use of these 0-links has been dev- 
ised by A. J. Perlis and C. Thornton. Their idea is to replace the O-links by 


pointers, called threads, to other nodes in the tree. These threads are constructed 
using the following rules: 


(1) A 0 rightChild field in node p is replaced by a pointer to the node that 
would be visited after p when traversing the tree in inorder. That is, it is 
replaced by the inorder successor of p. 


(2) A 0 leftChild link at node p is replaced by a pointer to the node that 
immediately precedes node p in inorder (i.e., it is replaced by the inorder 
predecessor of p). 


Figure 5.20 shows the binary tree of Figure 5.10(b) with its new threads 
drawn in as broken lines. This tree has 9 nodes and 10 0-links, which have been 
replaced by threads. If we traverse the tree in inorder, the nodes will be visited 
in the order H, D, 1, B, E, A, F, C, G. For example, node E has a predecessor 
thread that points to B and a successor thread that points to A. 

In the memory representation we must be able to distinguish between 
threads and normal pointers. This is done by adding two Boolean fields, feft- 
Thread and rightThread, to TreeNode. Let 1 be a pointer to a tree node. If 
tleftThread == true, then tleftChild contains a thread; otherwise it contains 
a pointer to the left child. Similarly if t—rightThread == true, then 
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pisces megs 


Figure 5.20: Threaded tree corresponding to Figure 5.10(b) 


trightChild contains a thread; otherwise it contains a pointer to the right child. 
Let ThreadedNode be the resulting node structure. 

In Figure 5.20 we see that two threads have been {eft dangling. One is the 
leftChild of H and the other the rightChild of G. In order that we leave no loose 
threads, we will assume a header node for all threaded binary trees. The original 
tee is the left subtree of the header node. An empty binary tree is represented by 
its header node as in Figure 5.21. The complete memory representation for the 
tree of Figure 5.20 is shown in Figure 5.22. 


leftThread leftChild data _rightChild rightThread 
| | 1: _ false 


boo ae 


Figure 5.21: An empty threaded binary tree 
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root 


Figure 5.22: Memory representation of threaded tree 


5.5.2 Inorder Traversal of a Threaded Binary Tree 


By using the threads, we can perform an inorder traversal without making use of 
a stack. Observe that for any node x in a binary tree, if x->rightThread == true, 
then the inorder successor of x is x-srightChild by definition of threads. Other- 
wise the inorder successor of x is obtained by following a path of left-child links 
from the right child of x until a node with leftThread == true is reached. Func- 
tion Next () (Program 5.13) finds and returns the inorder successor of node 
currentNode ina threaded binary tee. (We assume that ThreadedInorderTree is 
anested class of ThreadedTree. 

The interesting thing to note about function Next () is that it is now possible 
to find the inorder successor of any arbitrary node in a threaded binary tree 
without using an additional stack. Since the tree is the left subtree of the header 
node and because of the choice of rightThread = false for the header node, the 
inorder sequence of nodes in the original binary tree may be obtained by 
repeated invocations of the function Next () beginning with currentNode pointing 
to the tree header. The computing time for the traversal is readily seen to be 
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T* ThreadedInorderlterator::Next () 
{/ Return the inorder successor of currentNode in a threaded binary tree 
ThreadedNode <T > *temp = currentNode —yrightChild; 
if (!currentNode ~>rightThread) 
while (!remp leftThread) temp = temp leftChild; 
currentNode = temp; 
if (currentNode == root) return 0; 
else return &currentNode —data; 


} 


Program 5.13: Finding the inorder successor in a threaded binary wee 


O(n) for an n-node tree. 


5.5.3 Inserting a Node into a Threaded Binary Tree 


We now examine how to make insertions into a threaded tree. This will give us a 

function for growing threaded trees. We shall study only the case of inserting r 

as the right child of a node s. The case of insertion of a left child is given as an 

exercise. The cases for insertion are 

(1) If s has an empty right subtree, then the insertion is simple and diagrammed 
in Figure 5.23(a). 

(2) If the right subtree of s is not empty, then this right subtree is made the 
right subtree of r after insertion. When this is done, r becomes the inorder 
predecessor of a node that has a leftThread == true field, and consequently 
there is a thread which has to be updated to point to r. The node containing 
this thread was previously the inorder successor of s. Figure 5.23(b) illus- 
trates the insertion for this case. 


In both cases s is the inorder predecessor of r, The details are given in function 
InsertRight (Program 5.14). It is assumed that function /norderSucc(r) retums the 
inorder successor of r, using an algorithm similar to that used in Next (). 


EXERCISES 


1. Write a C++ function to insert a new node / as the left child of node s in a 
threaded binary tree. The left subtree of s becomes the left subtree of . 
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pose 


Lae See ea ee 


before (b) after 


Figure 5.23: Insertion of r as a right child of s in a threaded binary tree 


2. 


Write a C++ forward iterator for threaded binary trees. You should 
sequence through the nodes in inorder. 


Write a function to traverse a threaded binary tree in postorder. What are 
the time and space requirements of your function? 
Write a function to traverse a threaded binary tree in preorder. What are 
the time and space requirements of your method? 


Consider threading a binary wee using preorder threads rather than inorder 
threads as in the text. Which of the traversals can be done without the use 
of a stack? For those that can be performed without a stack, write a C++ 
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template <class T> 
void ThreadedTree <T >::InsertRight(ThreadedNode <T > *s, 
ThreadedNode <T > *r) 
{4 Insert r as the right child of s. 
r—rightChild = s >rightChild; 
r~rightThread = s >rightThread; 
r—leftChild = s3 
r—leftThread = true; // leftChild is a thread 
S—rightChild =r; 
5—rightThread = false; 
if (! r—srightThread) { 
ThreadedNode <T > *temp = InorderSucc (r); 
// returns the inorder successor of r 
temp —leftChild = r; 


} 
Program 5.14: Inserting r as the right child of s 


function and analyze its space complexity. 

6. Consider threading a binary tree using postorder threads rather than inorder 
threads as in the text. Which of the traversals can be done without the use 
of a stack? For those that can be performed without a stack, write an algo- 
rithm and analyze its space complexity. 


5.6 HEAPS 


5.6.1 Priority Queues 


Heaps are frequently used to implement priority queues. In this kind of queue, 
the element to be deleted is the one with highest (or lowest) priority. At any 
time, an element with arbitrary priority can be inserted into the queue. ADT 5.2 
specifies a max priority queue as a C++ abstract class. 

It is assumed that the type T is defined so that the C++ relational operators 
(<, >, etc.) compare element priorities. We represent MaxPQ using pure virtual 
functions (and hence make it an abstract class) because it can be realized bya 
number of different data structures. Without knowing which data structure will 
be used, it is impossible to implement the MaxPQ operations. However, we 
know that any data structure D that implements a max priority queue must 


280 Trees 


template <class T> 
class MaxPQ { 
public: 
virtual “MaxPQ() {} 
# virtual destructor 
virtual bool isEmpty () const = 0; 
3 # return true iff the Priority queue is empty 
virtual const T& Top () const = 0; 
1 return reference to max element 
virtual void Push(const T&)=0; 
__ Madd an element to the Priority queue 
virtual void Pop() = 0; 
# delete element with max priority 


‘ 


a a 
ADT 5.2: A max priority queue 


implement the operations specified in ADT 5.2. This is ensured by implement- 
ing D as a publicly derived class of MaxPQ. 


Example 5.1: Suppose that we are selling the services of a machine. Each user 
pays a fixed amount per use. However, the time needed by each user is different. 
We wish to maximize the retums from this machine under the assumption that 
the machine is not to be kept idle unless no user is available. This can be done 
by maintaining a priority queue of all persons waiting to use the machine. 
Whenever the machine becomes available, the user with the smallest time 
requirement is selected. Hence, a min priority queue is required. When a new 
user requests the machine, his/her request is put into the priority queue. 

If each user needs the same amount of time on the machine but people are 
willing to pay different amounts for the service, then a priority queue based on 
the amount of payment can be maintained. Whenever the machine becomes 


available, the user paying the most is selected. This requires a max priority 
queue. 


Example 5.2: Suppose that we are simulating a large factory. This factory has 
many machines and many jobs that require processing on some of the machines. 
An event is said to ovcur whenever a machine completes the processing of a job. 
When an event occurs, the job has to be moved to the queue for the next machine 
{if any) that it needs. If this queue is empty, the job can be assigned to the 
machine immediately. Also, a new job can be scheduled on the machine that has 
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become idle (provided that its queue is not empty). 

To determine the occurrence of events, a priority queue is used. This queue 
contains the finish time of all jobs that are presently being worked on. The next 
event occurs at the least time in the priority queue. So, a min priority queue can 
be used in this application. O 


The simplest way to represent a priority queue is as an unordered linear 
list. Regardless of whether this list is represented sequentially or as a chain, the 
dsEmpty function takes O(1) time; the Top () function takes O{n) time, where x is 
the number of elements in the priority queue; a push can be done in O(1) time as 
it doesn’t matter where in the list the new element is inserted; and a Pop takes 
©(n) time as me must first find the element with max priority and then delete it 
As we shall see shortly, when a max heap is used, the complexity of IsEmpty and 
Top is O(1) and that of Push and Pop is O(log n). 


5.6.2 Definition of a Max Heap 


In Section 5.2.2, we defined a complete binary tree. In this section we present a 
special form of a complete binary tree that is useful in many applications. 


Definition: A max (min) tree is a tree in which the key value in each node is no 
smaller (larger) than the key values in its children (if any). A max heap is acom- 
plete binary tree that is also a max tree. A min heap is a complete binary tree 
that is also a min tree. 0 


Some examples of max heaps and min heaps are shown in Figures 5.24 and 
5.25, respectively. 


PLE 


Figure 5.24: Max heaps 


From the definitions, it follows that the key in the root of a min tree is the 
smallest key in the tree, whereas that in the root of a max tree is the largest. 
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Figure 5.25: Min heaps 


When viewed as an ADT, a max heap is very simple. The basic operations are 
the same as those for a max priority queue (ADT 5.2). Since a max heap is a 
complete binary tree, we represent it using an artay heap. The private data 


members of the template class MaxHeap, which derives from MaxPQ <T>, are 
given below. 


private: 
T *heap; # element array 
int heapSize; 4 number of elements in heap 
int capacity; if size of the array heap 


Program 5.15 defines the MaxHeap constructor. A max heap is empty iff heap- 
Size == 0; the max element of a nonempty max heap is in heap [I]. Since the 
codes for /sEmpty and Top as fairly straightforward, these are omitted. 


template <class 7> 
MaxHeap<T>::MaxHeap (int theCapacity = 10) 
{ 
if (theCapacity < 1) throw “Capacity must be >= 1."; 
capacity = theCapacity; 
heapSize = 0; 
heap = new T [capacity +1]; // heap [0] is not used 


} 


Program 5.15: Max heap constructor 
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5.6.3 Insertion into a Max Heap 


A max heap with five elements is shown in Figure 5.26(2). When an element is 
added to this heap, the resulting six-element heap must have the structure shown 
in Figure 5.26(b), because a heap is a complete binary tree. To determine the 
correct place for the element that is being inserted, we use a bubbling up process 
that begins at the new node of the tree and moves toward the root. The element 
to be inserted bubbles up as far as is necessary to ensure a max heap following 
the insertion. If the element to be inserted has key value 1, it may be inserted as 
the left child of 2 (i.¢., in the new node). If instead, the key value of the new ele- 
ment is 5, then this cannot be inserted as the left child of 2 (as otherwise, we will 
not have a max heap following the insertion). So, the 2 is moved down to its left 
child (Figure 5.26(c)), and we determine if placing the 5 at the old position of 2 
results in a max heap. Since the parent element (20) is at Jeast as large as the 
element being inserted (5), it is all right to insert the new clement at the position 
shown in the figure. Next, suppose that the new element has value 2] rather than 
5. In this case, the 2 moves down to its left child as in Figure 5.26(c). The 21 
cannot be inserted into the old position occupied by the 2, as the parent of this 
position is smaller than 21. Hence, the 20 is moved down to its right child and 
the 21 inserted into the root of the heap (Figure 5.26(d)). 

To implement the insertion strategy just described, we need to go from an 
element to its parent. Lemma 5.4 enables us to locate the parent of any element 
easily. Program 5.16 performs an insertion into a max heap. 


Analysis of Push: The insertion function begins at a leaf of a complete binary 
tree and moves up toward the root. At each node on this path, O(1) amount of 
work is done. Since a complete binary tree with n elements has a height 
[loga(+1)], the while loop of the insertion function is iterated O(log 1) times. 
Hence, the complexity of Push is O(log 1), where n is the number of elements in 
the heap. O 


5.6.4 Deletion from a Max Heap 


When an element is to be deleted from a max heap, it is taken from the root of 
the heap. For instance, a deletion from the heap of Figure 5.26(d) results in the 
removal of the element 21. Since the resulting heap has only five elements in it. 
the binary tree of Figure 5.26(d) needs to be restructured to correspond to a com- 
plete binary tree with five elements. To do this, we remove the element in posi- 
tion 6 (i.e., the element 2). Now we have the right structure (Figure 5.27(a)), but 
the root is vacant and the element 2 is not in the heap. If the 2 is inserted into the 
root, the resulting binary tree is not a max heap. The element at the root should 


284 Trees 


Figure 5.26: Insertion into a max heap 


be the largest from among the 2 and the elements in the left and right children of 
the root. This element is 20. It is moved into the root, thereby creating a 
vacancy in position 3. Since this position has no children, the 2 may be inserted 
here. The resulting heap is shown in Figure 5.26(a). 

Now, suppose we wish to perform another deletion. The 20 is to be 
deleted. Following the deletion, the heap has the binary tree structure shown in 
Figure 5.27(b). To get this structure, the 10 is removed from position 5. It can- 
not be inserted into the root, as it is not large enough. The 15 moves to the root, 
and we attempt to insert the 10 into position 2. This is, however, smaller than the 
14 below it. So, the 14 is moved up and the 10 inserted into position 4. The 
resulting heap is shown in Figure 5.27(c). 

Program 5.17 implements this trickle down suategy to delete from a heap. 


Analysis of Pop: Since the height of a heap with n elements is [logz(n +)], the 
while loop of Program 5.17 is iterated O(log n) times. Each iteration of this loop 
takes O(1) time. Hence, the complexity of Pop is O(log n). 0 


template <class T> 
void MaxHeap<T>::Push(const T& ¢) 
{4 Insert ¢ into the max heap. 
if (heapSize == capacity) {// double the capacity 
ChangeSize 1D (heap, capacity, 2*capacity); 
capacity *= 2; 
int currentNode = ++heapSize; 
while (currentNode != | && heap [currentNode /2) < e) 
{/ bubble up 
heap [currentNode ) = heap (currentNode /2); /{ move parent down 
currentNode [= 2; 


heap [currentNode } = e; 


} 


Program 5.16: Insertion into a max heap 


eo (15) 
e 2) CeO 
(9) 


Figure 5.27: Detetion from a heap 


EXERCISES 


1. Compare the run-time performance of max heaps with that of unordered 
and ordered linear lists as a representation for priority queues. For this 
comparison, program the max heap push and pop algorithms, as well as 
algorithms to perform these tasks on unordered and ordered linear lists that 
are maintained as sequential lists in a one-dimensional array. Generate a 
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template <class T> 
void MaxHeap <T>::Pop() 
{# Delete max element, 


if (/sEmpty ()) throw “Heap is empty. Cannot delete."; 
heap{l).TQ; # delete max element 


4 temove last element from heap 
T lastE = heap {heapSize - -}; 


Hsickle down 
int currentNode = 1; root 


int child = 2; Hachild of currentNode 
while (child <= heapSize) 
{ 


Ht set child to larger child of currentNode 
if (child < heapSize && heap [child] < heap [child +1}) child++; 


# can we put lastE in currentNode? 
if (lastE >= heap [child}) break; // yes 


Hno 
heap {currentNode | = heap (child }; / move child up 
currentNode = child; child *= 2; H move down a level 


) 
heap (currentNode | = lastE; 


Program 5.17; Deletion from a max heap 


random sequence of n values and insert these into the priority queue. Next, 
perform a random sequence of m inserts and deletes starting with the initial 
queue of 1 values. This sequence is to be generated so that the next opera- 
tion in the sequence has an equal chance of being either an insert or a 
delete. Care should be taken so that the sequence does not cause the prior- 
ity queue to become empty at any time. Measure the time taken for the 
sequence of m operations using both a max heap and an unordered list. 
Divide the total time by m and plot the times as a function of n. Do this for 
n= 100, 500, 1000, 2000, 3000, and 4000. Set m to be 1000. Make some 
qualitative statements about the relative performance of the two representa- 
tions for a max priority queue. 
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2. Write a C++ abstract class similar to ADT 5.2 for the ADT MinPQ, which 
defines a min priority queue. Now write a C++ class MinHeup that derives 
from this abstract class and implements all the virtual functions of MinPQ. 
The complexity of each function should be the same as that for the 
corresponding function of MaxHeap. 


3. The worst-case number of comparisons performed during an insertion into 
a max heap can be reduced to O(loglog 7) by performing a binary search on 
the path from the new leaf to the root. This does not affect the number of 
data moves though. Write an insertion algorithm that uses this strategy. 
Redo Exercise 1 using this insertion algorithm. Based on your expeni- 
ments, what can you say about the value of this strategy over the one used 
in Program 5.16? ; 


5.7. BINARY SEARCH TREES 
5.7.1 Definition 


A dictionary is a collection of pairs, each pair has a key and an associated ele- 
ment. Although naturally occurring dictionaries have several pairs that have the 
same key, we make the assumption here that no two pairs have the same key. 
The data structure, binary search tree, that we study in this section is easily 
extended to accommodate dictionaries in which several pairs have the same key. 
ADT 5.3 gives the specification of a dictionary. The data type for the keys and 
elements are, respectively, K and E. 


template <class K, class E> 
class Dictionary { 
public: 
virtual bool /sEmpry () const = 0; 
# return true iff the dictionary is empty 
virtual pair <K, E>* Get (const K&) const = 0; 
# return pointer to the pair with specified key; return 0 if no such pair 
virtual void /nsert(const pair <K, E>&)=0; 
‘insert the given pair; if key is a duplicate update associated element 
virtual void Delete (const K&) = 0; 
# delete pair with specified key 
}s 


ADT 5.3: A dictionary 
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A binary search tree has a better performance than any of the data struc- 
tures studied so far when the functions to be performed are search (i.e. get), 
insert, and delete. In fact, with a binary search tree, these functions can be per- 
formed both by key value and by rank (i.e., find an element with key k; find the 
fifth smallest element; delete the element with key k; delete the fifth smallest ele- 
ment; insert an element and determine its rank; and so on). 


Definition: A binary search tree is a binary tree. It may be empty. If it is not 
empty then it satisfies the following properties: 


(Lt) Every element has a key and no two elements have the same key (i.c., the 
keys are distinct). 


(2) The keys (if any) in the left subtree are smaller than the key in the root.” 
(3) The keys (if any) in the right subtree are larger than the key in the root. 
(4) The left and right subtrees are also binary search trees. 


There is some redundancy in this definition. Properties (2), (3), and (4) 
together imply that the keys must be distinct. So, property (1) can be replaced by 
the property: The root has a key. 

Some examples of binary trees in which the elements have distinct keys are 
shown in Figure 5.28. In this figure, only the key component of each dictionary 
pair is shown. The tree of Figure 5.28(a) is not a binary search tree, despite the 
fact that it satisfies properties (1), (2), and (3). The right subtree fails to satisfy 
property (4). This subtree is not a binary search wee, as its right subtree has a 
key value (22) that is smaller than that in the subtree’s root (25). The binary 
trees of Figures 5.28(b) and (c) are binary search trees. 


@® @ 


Figure 5.28: Binary trees 
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5.7.2 Searching a Binary Search Tree 


Since the definition of a binary search tree is recursive, it is easiest to describe a 
Tecursive search method. Suppose we wish to search for an element with key k. 
We begin at the root. If the root is 0, then the search tree contains no elements 
and the search is unsuccessful. Otherwise, we compare k with the key in the 
root. If k is less than the key in the root, then only the left subtree is to be 
searched. If k is larger than the key in the root, only the right subtree needs to be 
searched. Otherwise, k equals the key in the root and the search terminates suc- 
cessfully. The subtrees may be searched recursively as in Program 5.18. This 
function assumes a linked representation for the search tree, which is assumed to 
be an object of the class BST. BST derives from the class Tree <pair <K, E> >. 
The recursion of Program 5.18 is easily replaced by a while loop, as in Program 
5.19. 


template <class K, class E> // Driver 
pair<K, E>* BST <K, E>::Get(const K& k) 
{// Search the binary search tree (*this) for a pair with key k. 
4 Mf such a pair is found, return a pointer to this pair; otherwise, return 0. 
return Get(roor, k); 


template <class K, class E> // Workhorse 
ee <K, E>* BST <K, E >::Get(TreeNode <pair <K, E> >* p, const K& k) 


if (!p) return 0; 

if (k < p data first) return Get (p >leftChild,k); 
if (k > p—>data. first) return Get (p—rightChild,k); 
return &p—data; 


} 


Program 5.18: Recursive search of a binary search tree 


We define the rank of a node to be its position in inorder; the first node 
visited in inorder has rank 1. If we wish to search by rank, each node should 
have an additional field /eftSize, which is one plus the number of elements in the 
left subtree of the node. For the search tree of Figure 5.28(b), the nodes with 
keys 2, 5, 30, and 40, respectively, have leftSize equal to 1, 2. 3, and |. Program 
5.20 searches for the rth smallest element. To reduce code clutter, we assume 
that TreeNode has an additional ficld called leftSize. As can be seen, a binary 
search tree of height 4 can be searched by key as well as by rank in O(h) time. 
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template <class K, class E> // Iterative version 
pe <K, E>* BST<K, E>::Get(const K& k) 
TreeNode <pair <K, E> > *currentNode = root; 
while (currentNode) 
if (k < currentNode data. first) 
currentNode = currentNode —leftChild; 
else if (k > currentNode data. first) 
currentNode = currentNode —>rightChild; 
else return &currentNode data; 


no matching pair 
return 0; 


} 


Program 5.19: Iterative search of a binary search tree 


template <class K, class E> // search by rank 

pair <K, E>* BST <K, E>::RankGet(int r) 

{// Search the binary search tree for the rth smallest pair. 

TreeNode <pair <K, E> > *currentNode = root; 

while (currentNode) y 
if (r < currentNode lefiSize) currentNode = currentNode —>leftChild; 
else if (r > currentNode —leftSize) 


{ 
r= currentNode —leftSize; 
currentNode = currentNode —rightChild; 


else return &currentNode data; 
return 0; 


} 


Program 5.20: Searching a binary search tree by rank 
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5.7.3 Insertion into a Binary Search Tree 


To insert a pair (k, e), we must first verify that its key is different from those of 
existing elements. To do this, a search is carried out, If the search is unsuccess- 
ful, then the element is inserted at the point the search terminated. For instance, 
to insert an element with key 80 into the tree of Figure 5.28(b), we first search for 
80. This search terminates unsuccessfully, and the last node examined is the one 
with key 40. The new pair is inserted as the right child of this node. The result- 
ing search tree is shown in Figure 5.29(a). Figure 5.29(b) shows the result of 
inserting a pair with the key 35 into the search tree of Figure 5.2%a). When the 
dictionary already contains a pair with key k, we simply update the element asso- 
ciated with this key to e. 


Figure 5.29: Inserting into a binary search tree 


Program 5.21 implements the insert strategy just described. If a node has a 
leftSize field, then this is to be updated too. Regardless, the insertion can be per- 
formed in O(A) time, where A is the height of the search tree. 


5.7.4 Deletion from a Binary Search Tree 


Deletion of a leaf element is quite easy. For example, to delete 35 from the tree 
of Figure 5.29(b), the left-child field of its parent is set to 0 and the node 
disposed. This gives us the tree of Figure 5.29(a). To delete the 80 from this 
tree, the right-child field of 40 is set to 0, obtaining the tee of Figure 5.28(b), and 
the node containing 80 is disposed. 

The deletion of a nonleaf element that has only one child is also easy. The 
node containing the element to be deleted is disposed, and the single-child takes 
the place of the disposed node. So, to delete the element 5 from the wee of 
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template <class K, class E> 
void BST <K, E>:Insert(const pair<K, E>& thePair) 
{i Insert the Pair into the binary search tree. 
4 search for thePair first, Pp is parent of p 
TreeNode <pair <K, E>> *p = root, *pp =0; 
while (p) { : 
PP =P; 
if (thePair. first < p -sdata. first) p = p—sleftChilds 
else if (the Pair. first > p ~sdata. first) p = p —rrightChild; 
else // duplicate, update associated element 
{p >data.second = thePair.second; return;} 


/ perform insertion 

p = new TreeNode <pair <K, E >> (thePair); 

if (root) # tree not empty 
if (thePair first < pp >data. first) pp—leftChild = p; 
else pp —>rightChild = p; 

else root = p; 


Program 5.21: Insertion into a binary search tree 


Figure 5,29(a), we simply change the pointer from the parent node (i.e., the node 
containing 30) to the single-child node (i.e., the node containing 2). 

When the element to be deteted is in a nonleaf node that has two children, 
the element is replaced by either the largest element in its left subtree or the 
smallest one in its right subtree. Then we proceed to delete this replacing ele- 
ment from the subtree from which it was taken. For instance, if we wish to 
delete the element with key 30 from the tree of Figure 5.29(a), then we replace it 
by either the largest element, 5, in its left subtree or the smallest element, 40, in 
its right subtree. Suppose we opt for the largest element in the left subtree. The 
5 is moved into the root, und the tree of Figure 5.30(a) is obtained. Now we must 
delete the second 5. Since this node has only one child, the pointer from its 
parent is changed to point to this child. The tree of Figure 5.30(b) is obtained. 
One may verify that regardless of whether the replacing element is the largest in 
the left subtree or the smailest in the right subtree, it is originally in a node with a 
degree of at most ane. So, deleting it from this node is quite easy. We leave the 
writing of the deletion function as an exercise. It should be evident thal a dele- 
tion can be performed in Of/t) time if the search tree has a height of 4. 
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(b) 


Figure 5.30: Deletion from a binary search tree 


5.7.5 Joining and Splitting Binary Trees 


Although search, insert, and delete are the operations most frequently performed 
on a binary search tree, the following additional operations are useful in certain 
applications: 


(a) 


(b) 


(c) 


ThreeWayJoin (small,mid,big): This creates a binary search tree consist- 
ing of the pairs initially in the binary search trees small and big, as well as 
the pair mid. It is assumed that each key in small is smaller than mid . first 
and that each key in big is greater than mid. first. Following the join, both 
small and big are empty. 

TwoWayJoin(small,big): This joins the two binary search trees small and 
big to obtain a single binary search tree that contains all the pairs originally 
in small and big. It is assumed that all keys of small are smaller than all 
keys of big and that following the join both smull and big are empty. 

Split (k,small,mid,big): The binary search mee *this is split into three 
parts: smail is a binary search tree that contains all pairs of *this that have 
key less than &; if *this contains a pair with key k, then this pair is returned 
in the reference parameter mid; big is a binary search tree that contains all 
pairs of *this that have key larger than k. Following the split operation 
*this is empty. 


A three-way join operation is particularly easy to perform. We simply 


obtain a new node and set its data field to mid, its left-child pointer to small.root, 
and its right-child pointer to big.root. This new node is made the root of *this. 
The time taken for this operation is O(1), and the height of the new tee is 
max {height (small), height (big)} + 1. 
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Consider the two-way join operation. If either smail or big is empty, the 
result is the other tree. When neither is empty, we may first delete from small the 
pair mid with the largest key. Let the resulting binary search tree be smatl’. To 
complete the operation, we perform the three-way join operation 
ThreeWayJoin (small’,mid,big). The overall time required to perform the two- 
way join operation is O(height (small)), and the height of the resulting tee is 
max({height (small), height(big)} + 1. The run time can be made 
O(min {height (small), height (big)}) if we retain with each tree its height. Then 
we delete the pair with the largest key from small if the height of smaif is no 
more than that of big; otherwise, we delete from big the pair with the smallest 
key. This is followed by a three-way join operation. 

To perform a split, we first make the following observation about splitting 
at the root (i.e., when k = root data . first). In this case, smail is the left subtree 
of *this, mid is the pair in the root, and big is the right subtree of +this. If k is 
smaller than the key at the root, then the root together with its right subtree is to 
be in big. When & is larger than the key at the root, the root together with its left 
subtree is to be in small. Using these observations, we can perform a split by 
moving down the search tree *this searching for a pair with key k. As we move 
down, we construct the two search trees small and big. The function to split 
*this given in Program 5.22. To simplify the code, we begin with two header 
nodes sHead and bHead for small and big, respectively. small is grown as the 
right subtree of sHead; big is grown as the left subtree of bHead. s (b) points to 
the node of sHead (bHead) at which further subtrees of *this that are to be part 
of small (big) may be attached. Attaching a subtree to small (big) is done as the 
right (left) child of s (b). 


Analysis of Split: The while loop maintains the invariant that all keys in the 
subtree with root currentNode are larger than those in the tree rooted at sHead 
and smaller than those in the tree rooted at bHead. The correctness of the func- 
tion is easy to establish, and its complexity is seen to be Otheight (+this)). One 
may verify that neither smal! nor big has a height larger than that of *this. 0 


5.7.6 Height of a Binary Search Tree 


Unless care is taken, the height of a binary search tree with n elements can 
become as large as n. This is the case, for instance, when Program 5.21 is used 
to insert the keys [1, 2, 3,..., ”], in this order, into an initially empty binary 
search tree. It can. however, be shown that when insertions and deletions are 
made at random using the functions given here, the height of the binary search 
tree is O(log 7) on the average. 

Search trees with a worst-case height of O(log n) are called balanced 
search trees. Balanced search tees that permit searches, inserts, and deletes to 
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template <class K, class E> 
void BST <K, E >::Split(const K& k, BST<K, E>& small, 
pair <K, E>*& mid, BST<K, E>& big) 
{// Split the binary search tree with respect to key k. 
if (!root) {small.root = big.root = 0; return;} / empty tree 
4 create header nodes for small and big 
TreeNode <pair <K, E>> *sHead = new TreeNode <pair<K, E> >, 
*5 = sHead, 
*bHead = new TreeNode<pair<K, E> >, 
*b = bHead, 
*currentNode = root; 
while (currentNode) 
if (k < currentNode data. first) {// add to big 
bleftChild = currentNode; 
b = currentNode; currentNode = currentNode —leftChild; 


} 
else if (k > currentNode ~sdata. first) {if add to small 
$s —rightChild = currentNode; 
5 = currentNode; currentNode = currentNode —rightChild; 


} 

else {// split at currentNode 
s—rightChild = currentNode SleftChild; 
b-leftChild = currentNode —rightChild; 
small root = sHead >rightChild; delete sHead; 
big.root = bHead >leftChild; delete bHead; 
mid = new pair <K, E >(currentNode —data. first, 

currentNode —data.second); 

delete currentNode; 
return; 


} 
no pair with key k 
5 —rightChild = b leftChild = 0; 
small. root = sHead —rightChild; delete sHead; 
big.root = bHead >leftChild; delete bHead; 
mid = 0, 
return; 


} 


Program 5.22 Splitting a binary search tree 
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be performed in 0) time exist. Most notable among these are AVL, red/black, 
2-3, 2-3-4, B, and B™ trees. These are discussed in Chapters 10 and 11. 


EXERCISES 


1. Write a C++ function to delete the pair with key k from a binary search 
tree. What is the time complexity of your function? 


2. Write a program to start with an initially empty binary search tree and 
make n random insertions. Use a uniform random number generator to 
obtain the values to be inserted. Measure the height of the resulting binary 
search tree and divide this height by logyn. Do this for n = 100, 500, 1000, 
2000, 3000, ---, 10,000. Plot the ratio height Aog,n as a function of n. 
The ratio should be approximately constant (around 2). Verify that this is 
$0, 

3. Suppose that each node in a binary search tree also has the field feftSize as 
described in the text. Write a function to insert a pair into such a binary 
search tree. The complexity of your function should be O(/), where h is 
the height of the search tree. Show that this is the case. 


4. Do Exercise 3, but this time write a function to delete the pair with the kth 
smallest key in the binary search tree. 


5. Write a C++ function that implements the three-way join operation in O(1) 
time. 


6. Write a C++ function that implements the two-way join operation in O(h) 
time, where A is the height of one of the two trees being joined. 


7. Any algorithm that merges together two sorted lists of size n and m, respec- 
tively, must make at least n + m — 1 comparisons in the worst case. What 
implications does this result have on the time complexity of any 
comparison-based algorithm that combines two binary search trees that 
have n and m pairs, respectively? 

8. In Chapter 7, we shall see that every comparison-based algorithm to sort 7 
elements must make O(nJog ») comparisons in the worst case. What impli- 
cations docs this result have on the complexity of initializing a binary 
search tree with # pairs? 


9. Notice that a binary search tree can be used to implement a priority queue. 
(a) Write a C++ class definition for a max priority queue that represents 
the priority queue as a binary search tree. This class should be a pub- 
licly derived class of MaxPQ. 
(b) Write a C++ function for all virtual functions of MaxPQ. Your codes 
for Top, Pop and Push should have complexity O(A), where is the 
height of the search tree. Since h is O(log n) on average, we can 
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perform each these priority queue operations in average time 
O(log 7). 

(c) Compare the actual performance of heaps and binary search trees as 
data structures for priority queues. For this comparison, generate 
random sequences of delete max and insert operations and measure 
the total time taken for each sequence by each of these data struc- 
tures. 


5.8 SELECTION TREES 


5.8.1 Introduction 


Suppose we have k ordered sequences, called runs, that are to be merged into a 
single ordered sequence. Each run consists of some records and is in nonde- 
creasing order of a designated field called the key. Let n be the number of 
records in all & runs together. The merging task can be accomplished by repeat- 
edly outputting the record with the smallest key. The smallest has to be found 
from & possibilities, and it could be the leading record in any of the k runs. The 
most direct way to merge & runs is to make k — 1 comparisons to determine the 
next record to output. For k > 2, we can achieve a reduction in the number of 
comparisons needed to find the next smallest element by using the selection tree 
data structure. There are two kinds of selection wees: winner trees and loser 
trees. 


5.8.2 Winner Trees 


A winner tree is a complete binary tree in which each node represents the 
smaller of its two children. Thus, the root node represents the smallest node in 
the tree. Figure 5.31 illustrates a winner tree for the case k= 8. 

The construction of this winner tree may be compared to the playing of a 
tournament in which the winner is the record with the smaller key. Then, each 
nonleaf node in the tree represents the winner of a tournament, and the root node 
represents the overall winner, or the smallest key. Each leaf node represents the 
first record in the corresponding run. Since the records being merged are gen- 
erally large, each node will contain only a pointer to the record it represents. 
Thus, the root node contains a pointer to the first record in run 4. 

A winner tree may be represented using the sequential allocation scheme 
for binary trees that results from Lemma 5.4. The number above each node in 
Figure 5.31 is the address of the node in this sequential representation. The 
record pointed to by the root has the smallest key and so may be output. Now, 
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ron2 cun3 run4 runS run6 run7 rmun8 


Figure 5.31: Winner tree for k = 8, showing the first three keys in each of the 
eight runs 


the next record from run 4 enters the winner tree. It has a key value of 15. To 
restructure the tree, the tournament has to be replayed only along the path from 
node iI to the root. Thus, the winner from nodes 10 and 11 is again node 11 (15 
< 20). The winner from nodes 4 and 5 is node 4 (9 < 19). The winner from 2 
and 3 is node 3 (8 < 9). The new tree is shown in Figure 5.32. The tournament is 
played between sibling nodes and the result put in the parent node. Lemma 5.4 
may be used to compute the address of sibling and parent nodes efficiently. Each 
new comparison takes place at the next higher level in the tee. 


Analysis of merging runs using winner trees: The number of levels in the tree 
is [ logo(k + 1)]. So, the time to restructure the tree is O{log2k). The tree has to 
be restructured each time a record is merged into the output file. Hence, the time 
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Figure 5.32: Winner tree of Figure 5.31 after one record has been output and the 
tree restructured (nodes that were changed are shaded) 


required to merge all n records is O(n log2k). The time required to set up the 
selection tree the first time is O(k). Thus, the total time needed to merge the k 
mns is O(n log2k). O 


5.8.3 Loser Trees 


After the record with the smallest key value is output, the winner tree of Figure 
5.31 is to be restructured. Since the record with the smallest key value is in run 
4, this restructuring involves inserting the next record from this run into the tree. 
The next record has key value 15. Tournaments are played between sibling 
nodes along the path from node 1] to the root. Since these sibling nodes 
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represent the losers of tournaments played earlier, we can simplify the restructur- 
ing process by placing in each nonleaf node a pointer to the record that loses the 
tournament rather than to the winner of the tournament. A selection tree in 
which each nonleaf node retains a pointer to the loser is called a loser tree. Fig- 
ure 5.33 shows the loser tree that corresponds to the winner tree of Figure 5.31. 
For convenience, each node contains the key value of a record rather than a 
Pointer to the record represented. The leaf nodes represent the first record in 
each run. An additional node, node 0, has been added to represent the overall 
winner of the tournament. Following the output of the overall winner, the tree is 
Testructured by playing tournaments along the path from node I to node 1. The 
records with which these tournaments are to be played are readily available from 


the parent nodes. As a result, sibling nodes along the path from 11 to 1 are not 
accessed. 


overall 
winner 


Figure 5.33: Loser tree corresponding to winner tree of Figure 5.31 
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EXERCISES 


I, Write C++ class definitions for winner and loser trees. 


2. Write a function to construct a winner tree for k records. Assume that k is a 
power of 2. Each node at which a tournament is played should store only a 
pointer to the winner. Show that this constniction can be carried out in 
time Ofk). 

3. Do Exercise 2 for the case when k is not restricted to being a power of 2. 


4. Write a function to construct a loser tree for k records. Use position 0 of 
your loser-tree array to store a pointer to the overall winner. Show that this 
construction can be carried out in time O(k). Assume that k is a power of 
2. 


5. Do Exercise 4 for the case when k is not restricted to being a power of 2. 


6. Write a function, using a tree of losers, to carry out a k-way merge of k 
runs, k > 2. Assume the existence of a function to initialize a loser tree in 
linear time. Show that if there are n>k records in all k runs together, then 
the computing time is O{nlog>k). 

7. Do the previous exercise for the case in which a tree of winners is used. 
Assume the existence of a function to initialize a winner tree in linear time. 


8. Compare the performance of your functions for the preceding two exer- 
cises for the case k = 8. Generate eight runs of data, each having 100 
records. Use a random number generator for this (the keys obtained from 
the random number generator will need to be sorted before the merge can 
begin). Measure and compare the time taken to merge the eight runs using 
the two strategies. 


5.9 FORESTS 


Definition: A forest is a set of n 2 0 disjoint trees. O 


A three-tree forest is shown in Figure 5.34. The concept of a forest is very close 
to that of a tree because if we remove the root of a tree, we obtain a forest. For 
example, removing the root of any binary tree produces a forest of two tees. In 
this section, we briefly consider several forest operations, including transforming 
a forest into a binary tree and forest traversals. In the next section, we use 
forests to represent disjoint sets. 
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5.9.1 Transforming a Forest into a Binary Tree 


Figure 5.34: Three-tree forest 


To wansform a forest into a single binary tree, we first obtain the binary tree 
representation of each of the trees in the forest and then link these binary trees 
together through the rightChild field of the root nodes. Using this transforma- 
tion, the forest of Figure 5.34 becomes the binary tree of Figure 5.35. 


Figure 5.35: Binary tree representation of forest of Figure 5.34 


We can define this transformation in a formal way as follows: 
Definition: 117, , --*, 7, is a forest of trees, then the binary tree corresponding 
to this forest, denoted by B(7,, °--, T,), 
{D isempty ifa =0 
(2) has root equal to root (71); has left subtree equal to B(T 1,712, °° *» Tim) 
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where 71), °°, Tim are the subtrees of root{7,);, and has right subtree 
B(T2, °>+,T,). 9 


5.9.2 Forest Traversals 


Preorder and inorder traversals of the corresponding binary tree T of a forest F 
have a natural correspondence to traversals on F. Preorder traversal of T is 
equivalent to visiting the nodes of F in forest preorder, which is defined as fol- 
lows: 


(1) If Fis empty then return. 

(2) Visit the root of the first tree of F. 

(3) Traverse the subtrees of the first tree in forest preorder. 
(4) Traverse the remaining trees of F in forest preorder. 


Inorder traversal of T is equivalent to visiting the nodes of F in forest inorder, 
which is defined as follows: 


(1) If F is empty then return. 

(2) Traverse the subtrees of the first tree in forest inorder. 
(3) Visit the root of the first wee. 

(4) Traverse the remaining trees in forest inorder. 


The proofs that preorder and inorder traversals on the corresponding binary 
tree are the same as preorder and inorder traversals on the forest are left as exer- 
cises. There is no natural analog for postorder traversal of the corresponding 
binary tree of a forest. Nevertheless, we can define the postorder traversal of a 
forest as follows: 


(1) If Fis empty then return. 

(2) Traverse the subtrees of the first tree of F in forest postorder. 
(3) Traverse the remaining trees of F in forest postorder. 

(4) Visit the root of the first tree of F. 


In a level-order traversal of a forest, nodes are visited by level, beginning 
with the roots of each tree in the forest. Within each level, nodes are visited 
from left to right. One may verify that the level-order traversal of a forest and 
that of its associated binary tree do not necessarily yield the same result. 
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EXERCISES 


1. Write a C++ template class definition for a forest of trees. 


2. Define the inverse transformation of the one that creates the associated 
binary tree from a forest. Are these transformations unique? 

3. Prove that the preorder traversal of a forest and the preorder traversal of its 
associated binary tree give the same result, 

4, Prove that the inorder traversal of a forest and the inorder traversal of its 
associated binary tree give the same result. 

5. Prove that the postorder traversal of a forest and that of its corresponding 
binary tree do not necessarily yield the same result. 

6. Prove that the level-order traversal of a forest and that of its corresponding 
binary tree do not necessarily yield the same result, 

Ts 


Write a nonrecursive function to traverse the associated binary tree of a 


forest in forest postorder. What are the time and space complexities of 
your function? 


8. Do the preceding exercise for the case of forest level-order traversal. 


5.10 REPRESENTATION OF DISJOINT SETS 
5.10.1 Introduction 


In this section we study the use of trees in the representation of sets. We shall 
assume that the elements of the sets are the numbers 0, 1, 2,3, --:,2—I. These 
numbers might, in practice, be indices into a symbol table where the actual 
names of the elements are stored. We shail assume that the sets being 
represented are pairwise disjoint (i.e., if S; and 5,, i#j, are two sets, then there is 
no element that is in both S; and S,). For example, when n = 10, the elements 
may be partitioned into three disjoint sets, S$; = {0, 6, 7, 8}, Sz = {1, 4, 9}, and 
Sy = {2,3. 5}. Figure 5.36 shows one possible representation for these sets. In 
this representation, each set is represented as a tree. Notice that for each set we 
have linked the nodes trom the children to the parent, rather than our usual 
method of linking from the parent to the children. The reason for this change in 
linkage will become apparent when we discuss the implementation of set opera- 
ons. 
The operations we wish to perform on these sets are: 


U)  Disjoint set union. If S, and S; are two disjoint sets, then their union S,US, 
= fall elements x such that x is in S; or S;}. Thus, 
SsUSs = 10.6.7. 8.4.4.9}. Since we have assumed that all sets are 
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Figure §.36: Possible tree representation of sets 


disjoint, we can assume that following the union of 5, and S;, the sets S; 
and S; do not exist independently; that is, they are replaced by 5;US; in the 
collection of sets. 

(2) Find(i). Find the set containing element i. Thus, 3 is in set Sy, and 8 is in 
set S). 


5.10.2. Union and Find Operations 


Let us consider the union operation first. Suppose that we wish to obtain the 
union of S$; and $2 (see Figure 5.36). Since we have linked the nodes from chil- 
dren to parent, we simply make one of the trees a subtree of the other. 5,;US 
could then have one of the representations of Figure 5.37. 


Figure 5.37: Possible representations of S$; US2 


To obtain the union of two sets, all that has to be done is to set the parent 
field of one of the roots to the other root. This can be accomplished easily if, 
with each set name, we keep a pointer to the root of the tree representing that set. 
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If, in addition, each root has a pointer to the set name, then to determine which 
set an element is currently in, we follow parent links to the root of its tree and 
use the pointer to the set name. The data representation for S;, S2, and S3 may 
then take the form shown in Figure 5.38. 


Set 
Name Pointer 
oe 


: Data representation for S,, 5, and $3 


In presenting the union and find algorithms we shall ignore the actual set 
names and just identify sets by the roots of the trees representing them. This will 
simplify the discussion. The transition to set names is easy. If we determine that 
element / is in a tree with root j, and j has a pointer to entry & in the set name 
table, then the set name is just name[k}. If we wish to unite sets S; and S;, then 
we wish to unite the wees with roots FindPointer(S;) and FindPointer(S;). Here 
FindPointer is a function that takes a set name and determines the root of the 
tree that represents it. This is done by an examination of the {set name, pointer] 
tuble, As we shall see, in many applications the set name is just the element at 
the root. The operation of Find(i) now becomes: Determine the root of the tree 
containing element i. The function Union(i,j) requires us to join the trees with 
roots i and j. Another simplifying assumption we shall make is that the set ele- 
ments are the numbers 0 through #—I. 

Since the set elements are numbered 0 through n-1, we represent the tree 
nodes using an array parent(n]. The ith element of this array represents the tree 
node that contains element i. This array element gives the parent pointer of the 
corresponding tree node. Figure 5.39 shows this representation of the sets, 5), 
S>, and S$ of Figure 5.36. Notice that root nodes have a parent of —-1. 

Program 5.23 contains the class definition and constructor for the data 
structure. 
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Figure 5.39: Array representation of S,, $2, and S3 of Figure 5.36 


class Sets { 
public: 
4 set operations follow 


private: 
int *parent; 
int n; // number of set elements 
h 
Sets ::Sets(int numberOfElements) 
if (numberOfElements < 2) throw "Must have at least 2 elements."; 
n= numberOfElements; 


parent = new int[n }; 
fill(parent, parent + n, -1); 


Program 5.23: Class definition and constructor for Sets 


‘We can now implement Find(i) by simply following the indices starting at i and 
continuing until we reach a node with parent value -1. For example, Find(5) 
starts at 5 and then moves to 5's parent, 2. Since parent[2} = -1, we have 
reached the root. The operation Union(i,j) is equally simple. We pass in two 
trees with roots / and j, i # j. Assuming that we adopt the convention that the first 
tree becomes a subtree of the second, the statement parent [i] = j accomplishes 
the union. Program 5.24 implements the union and find operations as just dis- 
cussed. 


Analysis of SimpleUnion and SimpleFind: Although these two functions are 
very easy to state, their performance characteristics are not very good. For 
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void Sets ::SimpleUnion(int i, int j) 
{i Replace the disjoint sets with roots i and j, i != j with their union. 
Parent [i] = j; 


int Sets ::SimpleFind(int i) 

{// Find the root of the tree containing element i. 
while (parent [i] >= 0) i = parent {i}; 
return i; 


} 


Program 5.24: Simple functions for union and find 


instance. if we start off with n elements each in a set of its own (i.e., S; = (i), 0S 
i <n), then the initial configuration consists of a forest with n nodes, and 
parent \i] =-1,0<i <n. Now let us process the following sequence of opera- 
tions: 


Union(0,1), Union(1,2), Union(2,3), Union(3,4), ++, Union(n - 2,n-1) 
Find(0), Find(1), +++, Find(n-1) 


This sequence results in the degenerate tree of Figure 5.40. Since the time taken 
for a union is constant, the 2 - 1 unions can be processed in time O(n). How- 
ever, each find operation requires following a sequence of parent pointers from 
the element to be found to the root. Since the time required to process a find for 
an element at level i of a tree is O(/), the total time needed to process the n finds 
is O(2¥.)f) = O(n). O 


We can improve performance by avoiding the creation of degenerate trees. 
In order to accomplish this we shall make use of a weighting rule for Union(i,j). 


Definition [Weighting rule for Union(ij)]: If the number of nodes in the tree 
with root f is less than the number in the tree with root j, then make the parent 
of i; otherwise make i the parent of j. 0 


When we use the weighting rule to perform the sequence of set unions 
given before, we obtain the trees of Figure 5.41. In this figure, the unions have 
been modified so that the input parameter values correspond to the roots of the 
trees to be combined. 

To implement the weighting rule, we need to know how many nodes there 
are in every tree. To do this easily, we maintain a count field in the root of every 
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Figure 5.40: Degenerate tree 
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Figure 5.41; Trees obtained using the weighting rule 
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tree. If i is a root node, then couni[i] equals the number of nodes in that tree. 
Since all nodes other than the roots of trees have a non-negative number in the 
parent field, we can maintain the count in the parent field of the roots as a nega- 
tive number. Initially, the parent field of all nodes contains —!. Using this con- 
vention, we obtain the union function of Program 5.25. 


void Sets s WeightedUnion(int i, int j) 
# Union sets with roots i and j, i#j, using the weighting rule. 
‘ parent {i] = count (i) and parent Ui) =-counr{j1. 


int temp = parent [i] + parent (j}; 

if (parent [i] > parent [j}) { // ihas fewer nodes 
parent (i) =j; 
parent {j] = temp; 


else { // jhas fewer nodes (or i and j have the same number of nodes) 
parent |j ql 
parent {i} = temp; 


} 


Program 5.25: Union function with weighting rule 


Analysis of WeightedUnion and SimpleFind: The time required to perform a 
union has increased somewhat but is still bounded by a constant (i.e., it is O(1)). 
The find function remains unchanged. The maximum time to perform a find is 
determined by Lemma 5.5. 


Lemma 5.5: Assume that we start with a forest of trees, each having one node. 
Let 7’ be a tree with m nodes created as a result of a sequence of unions each per- 
formed using function WeightedUnion. The height of T is no greater than 
[log, mJ + 1. 


Proof: The lemma is clearly true for # = 1. Assume it is true for all trees with i 
nodes, i $m — 1. We shalt show that it is also true for i = m. Let Tbe a tree with 
m nodes created by function WeightedUnion. Consider the last union operation 
performed, Union(k,j). Let a be the number of nodes in tree j and m—a the 
number in k. Without loss of generality we may assume 1 <a <m/2. Then the 
height of Tis either the same as that of k or is one more than that of j. If the 
former is the case, the height of Tis < [Jog2 (m-a)] + 1 < [logy m] + 1. If the 
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later is the case, the height of 7 is < |log,aj+2 $ |logym2} +2 < 
(log, mj +1.0 


Example 5.3 shows that the bound of Lemma 5.5 is achievable for some 
sequence of unions. 


Example 5.3: Consider the behavior of function WeightedUnion on the follow- 
ing sequence of unions starting from the initial configuration parent[i} = 
-count(iJ=~—1,0Si<n=8: 


Union(0,1), — Union(2,3),  Union(4,5), — Union(6,7), 
Union(0,2),  Union(4,6), — Union(0,4) 


The trees of Figure 5.42 are obtained. As is evident, the height of each tree 
with m nodes is [log, m] + 1.0 


From Lemma 5.5, it follows that the time to process a find is O(log m) if 
there are m elements in a tree. If an intermixed sequence of u ~1 union and f 
find operations is to be processed, the time becomes O(u + flog u), as no tree 
has more than u nodes in it. Of course, we need O(n) additional time to initialize 
the n-tree forest. 0 


Surprisingly, further improvement is possible. This time the modification 
will be made in the find algorithm using the collapsing rule. 


Definition (Collapsing rule}: If j is a node on the path from i to its root and 
Parent [i] # root (i), then set parent [j} to roor(i). O 


Function Collapsing Find (Program 5.26) incorporates the collapsing rule. 


Example 5.4: Consider the tree created by function WeightedUnion on the 
sequence of unions of Example 5.3. Now process the following eight finds: 


Find(7), Find(7), «++ , Find(7) 


If SimpleFind is used, each Find(7) requires going up three parent link fields for 
a total of 24 moves to process all eight finds. When CollapsingFind is used, the 
first Find(7) requires going up three links and then resetting two links. Note that 
even though only two parent links need to be reset, function Collapsing Find will 
actually reset three (the parent of 4 is reset to 0). Each of the remaining seven 
finds requires going up only one link field. The total cost is now only 13 moves. 
a 


Analysis of WeightedUnion and CollapsingFind: Use of the collapsing rule 
roughly doubles the time for an individual find. However, it reduces the worst- 
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(a) Height-4 tree following Union (0,4) 


Figure 5.42: Trees achieving worst-case bound 
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int Sets ::CollapsingFind(int i) 
{// Find the root of the tree containing element i. 

4 Use the collapsing rule to collapse all nodes from i to the root. 
for (int r = i; parent [r] >= 0; r = parent (r)); # find root 
while (i != r) {// collapse 

int s = parent [i]; 
parent[iJ=r; 
i=s; 


} 


return r; 


Program 5.26: Collapsing rule 


case time over a sequence of finds. The worst-case complexity of processing a 
Sequence of unions and finds using WeightedUnion and Collapsing Find is stated 
in Lemma 5.6. This lemma makes use of a function o(p,q) that is related to a 
eucuonal inverse of Ackermann’s function A (i,j). These functions are defined 
as follows: 


AQ j)=%, forj2 
AG, 1)=A(i-1,2) fori22 
AG/)=AG-LAGJ-D) fori, j22 


(p,q) = min{z 211 A(z,|p/q]) > logog), p2g2t 


The function A (i,j) is a very rapidly growing function. Consequently, o 
grows very slowly as p and q are increased. In fact, since A(3,1)= 16, 
(p,q) $ 3 for g < 2!© = 65,536 and p > g. Since A(4,1) is a very large number 
and in our application q will be the number, n, of set elements and p will be n + f 
(fis the number of finds), a(p,q) $ 4 for all practical purposes. 0 


Lemma 5.6 [Tarjan and Van Leeuwen]: Assume that we start with a forest of 
trees, each having one node. Let T(f,u) be the maximum time required to pro- 
cess any intermixed sequence of f finds and u unions. Assume that u 2 #/2. 
Then 

Kin + falftayn)) ST fu) < ko(a + f aft an) 
for some positive constants k, and k5. 0 
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The requirement that w > 2/2 in Lemma 5.6, is really not significant, as 
when u < /2, some elements are involved in no union operation. These ele- 
ments remain in singleton sets throughout the sequence of union and find opera- 
tions and can be eliminated from consideration, as find operations that involve 
these can be done in O(1) time each. Even though the function a(f,u) is a very 
slowly growing function, the complexity of our solution to the set representation 
problem is not linear in the number of unions and finds. The space requirements 
are one node for each element. 

In the exercises, we explore alternatives to the weight rule and the collaps- 
ing rule that preserve the time bounds of Lemma 5.6. 


5.10.3 Application to Equivalence Classes 


Consider the equivalence pairs processing problem of Section 4.9. The 
equivalence classes to be generated may be regarded as sets. These sets are dis- 
joint, as no polygon can be in two equivalence classes. Initially, all n polygons 
are in an equivalence class of their own; thus parent{i] = -1,0Si<n. If an 
equivalence pair, i = j, is to be processed, we must first determine the sets con- 
taining i and j. If these are different, then the two sets are to be replaced by their 
union. If the two sets are the same, then nothing is to be done, as the relation 
i =j is redundant, i and j are already in the same equivalence class. To process 
cach equivalence pair we need to perform two finds and at most one union. 
Thus, if we have n polygons and m equivalence pairs, we need to spend O(n) 
time to set up the initial n-tree forest, and then we need to process 2m finds and 
at most min{n ~ 1, m} unions. (Note that after 1 — 1 unions, all polygons will 
be in the same equivalence class and no more unions can be performed.) If we 
use WeightedUnion and CollapsingFind, the total time to process the 
equivalence relations is O(n + ma(2m, min{n—1, m})). Although this is slightly 
worse than the algorithm of Section 4.9, it needs less space and is on line. By 
“ton line,’ we mean that as each equivalence is processed, we can tell which 
equivalence class each polygon is in. 


Example 5.5: Consider the equivalence pairs example of Chapter 4. Initially, 
there are 12 trees, one for each variable. parenr{i] = -1,0<i< 12. The tree 
configuration following the processing of each equivalence pair is shown in Fig- 
ure 5.43. Each tree represents an equivalence class. It is possible to determine if 
two elements are currently in the same equivalence class at each stage of the 
processing simply by making (wo finds. 0 
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Figure 5.43: Trees for Example 5.5 


EXERCISES 


1. Suppose we start with n sets, each containing a distinct element. 


(a) | Show that if u unions are performed, then no set contains more than 
u + 1 elements. 


(b) Show that at most # — | unions can be performed before the number 


316 Trees 


(©) 


(d) 


of sets becomes 1. 


Show that if fewer than [2/2] unions are performed, then at least one 
set with a single element in it remains. 


Show that if « unions are performed, then at least max{m ~ 2u, 0} 
singleton sets remain. 


2. Using the result of Example 5.5, draw the trees after processing the instruc- 
tion wion(11,9), 


3. Experimentally compare the performance of SimpleUnion and SimpleFind 
(Program 5.24) with WeightedUnion (Program 5.25) and CollapsingFind 


(Program 5.26). For this, generate a random sequence of union and find 
operations, 


4. (a) 


(b) 


(c) 


(d) 


(b) 


‘Write a function HeightUnion that uses the height rule for union 
operations instead of the weighting rule. This rule is defined below: 


Definition (Height Rule): If the height of tee i is less than that of 
(ee j, then make j the parent of i, otherwise make i the parent of j. 0 
Your function must run in O(1) time and should maintain the height 
of each tree as a negative number in the parent field of the root. 

Show that the height bound of Lemma 5.5 applies to wees con- 
structed using the height rule. 

Give an example of a sequence of unions that start with singleton 
sets and create trees whose height equals the upper bound given in 
Lemma 5.5. Assume that each union is performed using the height 
rule. 

Experiment with functions WeightedUnion (Program 5.25) and 
HeightUnion to determine which one produces better results when 
used in conjunction with function Collapsing Find (Program 5.26). 
Write a function SplittingFind that uses path splitting for the find 
operations instead of path collapsing. This is defined below: 
Definition [Path Splitting]: In path splitting, the parent pointer in 
each node (except the root and its child) on the path from i to the root 
is changed to point to the node’s grandparent. 0. 


Note that when path splitting is used, a single pass from i to the root 
suffices. Tarjan and Van Leeuwen have shown that Lemma 5.6 holds 
when path splitting is used in conjunction with either the weight or 
height rule for unions. 


Experiment with functions CollapsingFind (Program 5.26) and Split- 
tingFind to determine which produces better results when used in 
conjunction with function WeightedUnion (Program 5.25). 
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6. (a) Write a function HalvingFind that uses path halving for the find 
operations instead of path collapsing. This is defined below: 


Definition [Path Halving}: In path halving, the parent pointer of 
every other node (except the root and its child) on the path from é to 
the root is changed to point to the node's grandparent. 0 


Note that path halving, like path splitting (Exercise 5) can be imple- 
mented with a single pass from / to the root. However, in path halv- 
ing, only half as many pointers are changed as in path splitting, Tar- 
jan and Van Leeuwen have shown that Lemma 5.6 holds when path 
halving is used in conjunction with either the weight or height rule 
for unions. 


(b) Experiment with functions CollapsingFind and HalvingFind to 
determine which one produces better results when used in conjunc- 
tion with function WeightedUnion. 


5.11 COUNTING BINARY TREES 


As a conclusion to our chapter on trees, we consider three disparate problems 
that amazingly have the same solution. We wish to determine the number of dis- 
tinct binary tees having n nodes, the number of distinct permutations of the 
numbers from | through 7 obtainable by a stack, and the number of distinct ways 
of multiplying n + 1 matrices. Let us begin with a quick look at these problems. 


5.11.1 Distinct Binary Trees 


We know that if 2 = 0 or » = 1, there is only one binary tree. If n = 2, then there 
are two distinct trees (Figure 5.44), and if n = 3, there are five such trees (Figure 
5.45). How many distinct trees are there with n nodes? Before deriving a solu- 
tion, we will examine the two remaining problems. You might attempt to sketch 
out a solution of your own before reading further. 


5.11.2 Stack Permutations 


In Section 5.3, we introduced preorder, inorder, and postorder uraversals and 
indicated that each traversal requires a stack. Suppose we have the preorder 
sequence A B C DEF GH / and the inorder sequence BCA ED GH F J of the 
same binary tree. Does such a pair of sequences uniquely define a binary tree? 
Put another way, can this pair of sequences come from more than one binary 
tree? 

To construct the binary tree from these sequences, we look at the first letter 
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Figure 5.44: Distinct binary trees with n = 2 


Pea £ 


Figure 5.45: Distinct binary trees with n = 3 


in the preorder sequence, A. This letter must be the root of the tree by definition 
of the preorder traversal (VLR). We also know by definition of the inorder traver- 
sal (LVR) that all nodes preceding A in the inorder sequence (B C) are in the left 
subtree, and the remaining nodes (E D G H F J) are in the right subtree. Figure 
5.46(a) is our first approximation to the correct tree. 

Moving right in the preorder sequence, we find B as the next root. Since no 
node precedes B in the inorder sequence, B has an empty left subtree, which 
means that C is in its right subtree. Figure 5.46(b) is the next approximation. 
Continuing in this way, we arrive at the binary tree of Figure 5.47(a). By formal- 
izing this argument (see the exercises), we can verify that every binary tree has a 
unique pair of preorder/inorder sequences. 
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Figure 5.46: Constructing a binary tree from its inorder and preorder sequences 


Let the nodes of an n-node binary tree be numbered from 1 through n. The 
inorder permutation defined by such a binary tree is the order in which its nodes 
are visited during an inorder traversal of the tree. A preorder permutation is 
similarly defined. 

As an example, consider the binary tree of Figure 5.47(a) with the node 
numbering of Figure 5.47(b). Its preorder permutation is 1, 2, «++, 9, and its 
inorder permutation is 2, 3, 1,5, 4, 7, 8, 6, 9. 

If the nodes of the tree are numbered such that its preorder permutation is 
1, 2, +--,m, then from our earlier discussion it follows that distinct binary trees 
define distinct inorder permutations. Thus, the number of distinct binary trees is 
equal to the number of distinct inorder permutations obtainable from binary trees 
having the preorder permutation, 1, 2, «:-,n. 

Using the concept of an inorder permutation, we can show that the number 
of distinct permutations obtainable by passing the numbers | through » through a 
stack and deleting in all possible ways is equal to the number of distinct binary 
trees with n nodes (see the exercises). If we start with the numbers 1, 2, and 3, 
then the possible permutations obtainable by a stack are 


(1, 2, 3) (1, 3, 2) (2, 1, 3) (2, 3, 1) 3,2, I) 


Obtaining (3, 1, 2) is impossible. Each of these five permutations corresponds to 
one of the five distinct binary trees with three nodes (Figure 5.48). 
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Figure 5.47: Binary tree constructed from its inorder and preorder sequences 
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Figure 5.48: Binary trees corresponding to five permutations 


$.11.3 Matrix Multiplication | 
= 


Another problem that surprisingly has a connection with the previous two 
involves the product of n matrices. Suppose that we wish to compute the product 
of matrices: 


My *My*---*Ms 


Since matrix multiplication is associative, we can perform these multiplications 
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in any order. We would like to know how many different ways we can perform 
these multiplications. For example, if n = 3, there are two possibilities: 


(M, *M2)* My 
M, * (M2 * M3) 


and if n = 4, there are five: 


(| * M2)*Ma)* My 
(i, * (Mz * M3))* Mg 
M, + (M2 *M3)* M4) 
(M | * (M2 * (M3 * Ma) 
(M4, * M2) * (M3 *M4)) 


Let 5, be the number of different ways to compute the product of matrices. 
Then b2 = 1,63 =2, and b4 =5. Let My, iS j, be the product M; * Mj,, * “"" * 
M,. The product we wish to compute is M,. We may compute M, by comput- 
ing any one of the products M1; * Mj41,., 1 $iSn. The number of distinct ways 
to obtain M,; and M;,),, are b; and b,_;, respectively. Therefore, letting b, = 1, 
we have 


not 
by, = L 5; dyin > 1 
1 
If we can determine the expression for 6, only in terms of n, then we have a solu- 
tion to our problem. 

Now instead let b, be the number of distinct binary trees with n nodes. 
Again an expression for b,, in terms of n is what we want. Then we see that b,, is 
the sum of all the possible binary trees formed in the following way: a root and 
two subtrees with 5; and b,_;_, nodes, for 0 Si <n (Figure 5.49). This explana- 
tion says that 


zt 
by = D5; ba-i-1 snl, and bg = 1 (5.3) 
i=0 
This formula and the previous one are essentially the same. Therefore, the 
number of binary trees with m nodes, the number of permutations of | to n obtain- 
able with a stack, and the number of ways to multiply n + 1 matrices are all 
equal. 


5.11.4 Number of Distinct Binary Trees 


To obtain the number of distinct binary trees with n nodes, we must solve the 
recurrence of Eq. (5.3). To begin we let 
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Figure 5.49: Decomposing 5, 
Bix) = Fx! (5.4) 
220 
which is the generating function for the number of binary trees. Next observe 
that by the recurrence relation we get the identity 


xB2(x) = B(x)-1 
Using the formula to solve quadratics and the fact that B(Q) = bo = 1 
(Eq.(5.3)), we get 
1~Vi-4x 


2x 
We can use the binomial theorem to expand (1 — 4x)!” to obtain 


Bax)= 


Ba)=d (-z["") ey} =Elwesl cry" 241 x" (5.5) 
Comparing Eqs. (5.4) and (5.5), we see that b,, which is the coefficient of x” in 
B(x), is 


2 


we if (nyt 2241 


Some simplification yields the more compact form 


eae en 
bi Seat {n 


which is approximately 
b,, = O(4"/n*?) 
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EXERCISES 


1. 


2: 


3. 


Prove that every binary tree is uniquely defined by its preorder and inorder 
sequences. 

Do the inorder and postorder sequences of a binary tree uniquely define the 
binary tree? Prove your answer. 

Do the inorder and preorder sequences of a binary tree uniquely define the 
binary tree? Prove your answer. 

Do the inorder and level-order sequences of a binary wee uniquely define 
the binary tee? Prove your answer. 

Write an algorithm to construct the binary tree with given preorder and 
inorder sequences. 

Repeat Exercise 5 with the inorder and postorder sequences. 

Prove that the number of distinct permutations of 1,2, ---, 1 obtainable by 
a stack is equal to the number of distinct binary trees with n nodes. (Hint: 
Use the concept of an inorder permutation of a tree with preorder permuta- 
tion 1,2, ---,m). 


5.12 REFERENCES AND SELECTED READINGS 

For more on trees, see The Art of Computer Programming: Fundamental Algo- 
rithms, Third Edition, by D. Knuth, Addison-Wesley, Reading, MA, 1998 and 
“*Handbook of data structures and applications,” edited by D. Mehta and S. 
Sahni, Chapman & Hall/CRC, Boca Raton, 2005. 


CHAPTER 6 


Graphs 


6.1 THE GRAPH ABSTRACT DATA TYPE 


6.1.1 Introduction 


The first recorded evidence of the use of graphs dates back to 1736, when 
Leonhard Euler used them to solve the now classical K6nigsberg bridge prob- 
lem. In the town of Konigsberg (now Kaliningrad) the river Pregel (Pregolya) 
flows around the island Kneiphof and then divides into two, There are, therefore, 
four land areas that have this river on its borders (see Figure 6.1(a)). These land 
areas are interconnected by seven bridges labeled a—g. The land areas them- 
selves are labeled A-D. The Konigsberg bridge problem is to determine 
whether, starting at one land area, it is possible to walk across all the bridges 
exactly once in returning to the starting land area. One possible walk is 


« start from land area B 
* walk across bridge a to island A 
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. take bridge e to area D 
¢ take bridge gto C 
° take bridge dto A 
¢ take bridge 6 to B 
. take bridge fto D 


This walk does not go across all bridges exactly once, nor does it return to the 
starting land area B. Euler answered the Kénigsberg bridge problem in the nega~ 
tive: The people of Konigsberg will not be able to walk across each bridge 
exactly once and return to the starting point. He solved the problem by 
Tepresenting the land areas as vertices and the bridges as edges in a graph (actu- 
ally a multigraph) as in Figure 6.1(b). His solution is elegant and applies to all 
graphs. Defining the degree of a vertex to be the number of edges incident to it, 
Euler showed that there is a walk starting at any vertex, going through each edge 
exactly once and terminating at the start vertex iff the degree of each vertex is 
even. A walk that does this is called Eulerian. There is no Eulerian walk for the 
KGnigsberg bridge problem, as all four vertices are of odd degree. 

Since this first application, graphs have been used in a wide variety of 
applications. Some of these applications are: analysis of electrical circuits, 
finding shortest routes, project planning, identification of chemical compounds, 
statistical mechanics, genetics, cybemetics, linguistics, social sciences, and so 
on. Indeed, it might well be said that of all mathematical structures, graphs are 
the most widely used. 


6.1.2 Definitions 


A graph, G, consists of two sets, Vand E. Vis a finite, nonempty set of vertices. 
E is a set of pairs of vertices; these pairs are called edges. V(G) and E(G) will 
represent the sets of vertices and edges, respectively, of graph G. We will also 
write G = (V,E) to represent a graph. In an undirected graph the pair of vertices 
representing any edge is unordered. Thus, the pairs («.v) and (»,«) represent the 
same edge. In a directed graph each edge is represented by a directed pair 
<u,v>, u is the tail and v the head of the edge”. Therefore, <v,u> and <u,v> 
represent two different edges. Figure 6.2 shows three graphs: G;, G2, and G3. 
The graphs G} and G» are undirected. Gs is a directed graph. 
The set representation of each of these graphs is 


*Often, both the undirected edge (i,j) and the directed edge <i,j > are writen as (i,j) 
Which is meant is deduced from the context. In this book, we refrain from this practice. 
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graph 


1) = 10,1,2,3}; E(G,) = {(0,1),(0,2),(0,3),(1,2),(1,3),(2,3)) 
VG = (0,1,2,3.4,5,6};  E(G2) = {(0,1),(0,2),(1,3),(1,4),(2,5),(2,6)} 
3) = {01,2}; E(G3) = (<0,1>,<1,0>,<1,2>}. 


Notice that the edges of a directed graph are drawn with an arrow from the 
tail to the head. The graph G> is a tree; the graphs G, and G3 are not. 

Since we define the edges and vertices of a graph as sets, we impose the 
following restrictions on graphs: 


{1) A graph may not have an edge from a vertex, v, back to itself. That is, 
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@G, 


Figure 6.2: Three sample graphs 


edges of the form (v, v) and <y, v> are not legal. Such edges are known as 
self edges or self loops. If we permit self edges, we obtain a data object 
referred to as a graph with self edges. An example is shown in Figure 
6.3(a). 

(2) A graph may not have multiple occurrences of the same edge. If we remove 
this restriction, we obtain a data object referred to as a multigraph (see Fig- 


ure 6.3(b)). 


(a) oe with a self edge Se 


Figure 6.3: Examples of graphlike structures 


The number of distinct unordered pairs (u,v) with u #v in a graph with n 
vertices is n(n — 1)/2. This is the maximum number of edges in any n-vertex, 
undirected graph. An n-vertex, undirected graph with exactly 2(n — 1)/2 edges 
is said to be complete. The graph G, of Figure 6.2(a) is the complete graph on 
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four vertices, whereas G2 and G3 are not complete graphs. In the case of a 
directed graph on # vertices, the maximum number of edges is n(n — 1). 

If (xv) is an edge in E(G), then we shall say the vertices u and v are adja- 
cent and that the edge (u,v) is incident on vertices u and v. The vertices adjacent 
to vertex I in G2 are 3, 4, and 0. The edges incident on vertex 2 in G2 are (0,2), 
(2,5), and (2,6). If <u,v> is a directed edge, then vertex u is adjacent to v, and v 
is adjacent from u. The edge <u,v> is incident to u and y. In G3, the edges 
incident to vertex | are <0,1>, <1,0>, and <1,2>. 

___ A subgraph of Gis a graph G’ such that V(G }.¢ V(G) and E(G .¢ E(G). 
Figure 6.4 shows some of the subgraphs of G and G3. 


1D} 1 


(0) (ii) (iii) 
(a) Some of the subgraphs of G, 


© 0) ©) 
© oO 
) @ 


(0) (ii) (ii) (iv) 
(b) Some of the subgraphs of G3 


Figure 6.4: Some subgraphs 


A path trom vertex u to vertex v in graph G is a sequence of vertices u, i), 
fa, +++, dg, ¥ such that (w. i)), (i), 42), --*, Gi. ¥) are edges in E(G). If G* is 
directed, then the path consists of <u,i,;>, <ij,i2>, *'', <ig, ¥> edges in 
E(G’). The fength of a path is the number of edges on it. A simple path is a 
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path in which all vertices except possibly the first and last are distinct. A path 
such as (0,1), (1,3), (3,2), is also written as 0,1,3,2. Paths 0,1,3,2 and 0,1,3,! of 
G, are both of length 3. The first is a simple path; the second is not. 0,1,2 is a 
simple directed path in G3. 0,1,2,1 is not a path in G3, as the edge <2,1> is not 
in E(G3). 

A cycle is a simple path in which the first and last vertices are the same. 
0,1,2,0 is a cycle in G,. 0,1,0 is a cycle in G3. For the case of directed graphs 
we normally add the prefix ‘‘directed”’ to the terms cycle and path, 

In an undirected graph, G, two vertices u and v are said to be connected iff 
there is a path in G from u to v (since G is undirected, this means there must also 
be a path from v to u). An undirected graph is said to be connected iff for every 
pair of distinct vertices u and v in V(G) there is a path from u to vin G. Graphs 
G, and G» are connected, whereas G4 of Figure 6.5 is not. A connected com- 
ponent (or simply a component), H, of an undirected graph is a maximal con- 
nected subgraph. By maximal, we mean that G contains no other subgraph that 
is both connected and properly contains H. G4 has two components, H, and H 
(see Figure 6.5). 


H, © (4) Hz 
| (3) (6) 
@ 


Ga 


Figure 6.5: A graph with two connected components 


A tree is a connected acyclic (i.e., has no cycles) graph. 

A directed graph G is said to be strongly connected iff for every pair of dis- 
tinct vertices and v in V(G), there is a directed path from u to v and also from v 
tou. The graph G; is not strongly connected, as there is no path from vertex 2 to 
1. A strongly connected component is a maximal subgraph that is strongly con- 
nected. Gy has two strongly connected components (see Figure 6.6). 

The degree of a vertex is the number of edges incident to that vertex. The 
degree of vertex 0 in G, is 3. If G is a directed graph, we define the in-degree of 
a vertex v to be the number of edges for which v is the head. The out-degree is 
defined to be the number of edges for which v is the tail. Vertex | of G3 has in- 
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Figure 6.6: Strongly connected components of G3 


degree 1, out-degree 2, and degree 3. If d; is the degree of vertex i in a graph G 
with n vertices and e edges, then the number of edges is 


a-) 
e=(Lay2 
t=O 


In the remainder of this chapter, we shall refer to a directed graph as a digraph. 
When we use the term graph, we assume that it is an undirected graph. Now that 
we have defined all the terminology we will need, let us consider the graph as an 
ADT. The graph data structure is specified as an abstract class in ADT 6.1. We 
use an abstract class, because many different representations for a graph are pos- 
sible. Classes that actually implement a graph using a specific graph representa- 
tion can derive publically from the abstract class Graph. 

‘The operations in ADT 6.1 are a basic set in that they allow us to create 
any arbitrary graph and do some elementary tests. In later sections of this 
chapter we will see functions that traverse a graph (depth-first or breadth-first 


seatch) and that determine if a graph has special properties (e.g., connected, 
biconnected, etc.}. 


6.1.3 Graph Representations 


Although several representations for graphs are possible, we shall study only the 
three most commonly used: adjacency matrices, adjacency lists, and adjacency 
multilists. Once again, the choice of a particular representation will depend 
upon the application one has in mind and the functions one expects to perform 
‘on the graph. 
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class Graph 
{# objects: A nonempty set of vertices and a set of undirected edges, 
H where each edge is a pair of vertices. 
public: 
virtual “Graph () {} 
4 virtual destructor 
bool /sEmpty () const {return n == 0}; 
‘4 ccturn true iff graph has no vertices 
int NumberOfVertices() const {return 7}; 
4 return number of vertices in the graph 
int NumberOfEdges() const {return e}; 
/ return number of edges in the graph 
virtual int Degree (int u) const = 0; 
4 return number of edges incident to vertex u 
virtual bool ExistsEdge (int u, int v) const = 0; 
# return true iff graph has the edge (u,v) 
virtual void InsertVertex(int v) = 0; 
4 insert vertex v into graph; v has no incident edges 
virtual void /nsertEdge(int u, int v) = 0; 
4 insert edge (u,v) into graph 
virtual void DeleteVertex(int v) = 03 
4 delete v and all edges incident to it 
virtual void DeleteEdge(int u, int v) = 0; 
Ht delete edge (u, v) from the graph 


private: 
int n; # number of vertices 
int e; “ number of edges 


+H 


ADT 6.1: Abstract data type Graph 


6.1.3.1 Adjacency Matrix 


Let G = (V, E) be a graph with n vertices, n 21. The adjacency matrix of G is a 
two-dimensional n xn array, say a, with the property that a[/][j] = 1 iff the edge 
G, j) (<i, j> for a directed graph) is in E(G). @{i][j] = 0 if there is no such edge 
in G. The adjacency matrices for the graphs G, , G3, and Gy are shown in Fig- 
ure 6.7, The adjacency matrix for an undirected graph is symmetric, as the edge 
(i, j) is in E(G) iff the edge (, i) is also in E(G). The adjacency matrix for a 
directed graph may not be symmetric (as is the case for G;). The space needed 
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( represent a graph using its adjacency matrix is n* bits. About half this space 
can be saved in the case of undirected graphs by storing only the upper or lower 
triangle of the matrix. 


01234567 
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1ji 0010000 

2}10010000 
0-123 3}0 1100000 
ojo rid OF 2 4100000100 
11011 0/0 10 5}00001010 
2b tol 11 01 600000101 
3)1 110 210 00 7300000010 
{a)G, (b) G3 (c) Ga 


Figure 6.7: Adjacency matrices 


From the adjacency matrix, one may readily determine if there is an edge 
connecting any two vertices i and j. For an undirected graph the degree of any 
vertex iis its row sum: 


Sata) 
j=0 


For a directed graph the row sum is the out-degree, and the column sum is the 
in-degree. 

Suppose we want to answer a nontrivial question about graphs, such as, 
How many edges are there in G? or, Is G connected? Adjacency matrices will 
require at least O(n?) time, as n? ~ n entries of the matrix (diagonal entries are 
zero) have to be examined. When graphs are sparse (i.e., most of the terms in the 
adjacency matrix are zero) one would expect that the former question could be 
answered in significantly less time, say O(e + n), where e is the number of edges 
in G, and e<<n7/2. Such a speed-up can be made possible through the use of a 
representation in which only the edges that are in G are explicitly stored. This 
Jeads to the next representation for graphs, adjacency lists. 
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adjLists data link 
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Figure 6.8: Adjacency lists 
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6.1.3.2 Adjacency Lists 


In this representation of graphs, the n rows of the adjacency matrix are 
represented as 1 chains (though sequential lists could be used just as well). 
There is one chain for each vertex in G. The nodes in chain i represent the ver- 
ices that are adjacent from vertex i, The data field of a chain node stores the 
index of an adjacent vertex. The adjacency lists for G,, G3, and G4 are shown in 
Figure 6.8. Notice that the vertices in each chain are not required to be ordered. 
An array adjLists is used so that we can access the adjacency list for any vertex 
in O(1) time. This array may be declared as 


Chain<int> *adjLists; 


where Chain is the template chain class of Chapter 4. If LinkedGraph is our 
class for undirected graphs represented as linked adjacency lists as in Figure 6.8, 


then adjLists is a private data member of this class and the class constructor is as 
below. 


LinkedGraph(const int vertices = 0) : n (vertices ), e (0) 
{adjLists = new Chain<int>[n];} 


For an undirected graph with n vertices and e edges, the linked adjacency 
lists representation requires an array of size n and 2e chain nodes. Each chain 
node has two fields. In terms of the number of bits of storage needed, the node 
count should be multiplied by log 7 for the array positions and log n + log e for 
the chain nodes, as it takes O(log m) bits to represent a number of value m. If 
instead of chains, we use sequential lists, the adjacency lists may be packed into 
an integer array node [n + 2e + 1]. In one possible sequential mapping , node {i] 
gives the starting point of the list for vertex i, OSi<n, and node [n] is set to 
n+2e +1. The vertices adjacent from vertex i are stored in node[i], ---, 
node {i + 1]-1, OSi <n. Figure 6.9 shows the representation for the graph G4 
of Figure 6.5. 


int nodes [n + 2%e + 1]; 


012.3 4 5 6 7 8 9 1011 12 13 14 15 16 17 18 19 20 21 22 
[ 9r11i3]ts}17°18[20]22[23] 2 [173 [00] 3[1[2)5 [6/4] 5] 716 


Figure 6.9: Sequential representation of graph G4 


The degree of uny vertex in an undirected graph may be determined by just 
counting the number of nodes in its adjacency list. 
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For a digraph, the number of list nodes is only e. The out-degree of any 
vertex may be determined by counting the number of nodes on its adjacency list. 
Determining the in-degree of a vertex is a little more complex. If there is a need 
to access repeatedly all vertices adjacent to another vertex, then it may be worth 
the effort to keep another set of lists in addition to the adjacency lists. This set of 
lists, called inverse adjacency lists, will contain one list for each vertex. Each 
list will contain a node for each vertex adjacent to the vertex it represents (see 
Figure 6.10). 


Figure 6.10: Inverse adjacency lists for G3 (Figure 6.2(c)) 


Alternatively, one can adopt a simplified version of the list structure used 
for sparse matrix representation in Chapter 4. Figure 6.11 shows the resulting 
structure for the graph G3 of Figure 6.2(c). The header nodes are stored sequen- 
tially. The first two fields in each node give the head and tail of the edge 
oe by the node, the remaining two fields are links for row and column 
chains. 


6.1.3.3 Adjacency Multilists 


In the adjacency-list representation of an undirected graph, each edge (u,v) is 
tepresented by two entries, one on the list for « and the other on the list for v. As 
we shall see, in some situations it is necessary to be able to determine the second 
entry for a particular edge and mark that edge as having been examined. This 
can be accomplished easily if the adjacency lists are actually maintained as mul- 
Ulists (i.e., lists in which nodes may be shared among several lists). For each 
edge there will be exactly one node, but this node will be in two lists (i-e., the 
adjacency lists for each of the two nodes to which it is incident). The new node 
structure is 
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Figure 6.11: Orthogonal list representation for G3 of Figure 6.2(c) 


[m | vertext | vertex? | tinki 


where m is a Boolean mark field that may be used to indicate whether or not the 
edge has been examined. The storage requirements are the same as for normal 
adjacency lists, except for the addition of the mark bit m. Figure 6.12 shows the 
adjacency multilists for G, of Figure 6.2(a). 


6.1.3.4 Weighted Edges 


In many applications, the edges of a graph have weights assigned to them. These 
weights may represent the distance from one vertex to another or the cost of 
going from one vertex to an adjacent vertex. In these applications, the adjacency 
matrix entries a[i][j] would keep this information too. When adjacency lists are 
used, the weight information may be kept in the list nodes by including an addi- 
tional field, weight. A graph with weighted edges is called a network, 


6.1.3.5 C++ Graph Classes 


There are two graph types—undirected and directed. A graph of each type may 
be weighted or unweighted and may be represented using any of the four 
methods—matrix, linked adjacency lists, sequential adjacency lists, adjacency 
multilists. So, to accomodate all of these possibilities, we must implement 16 
graph classes. For example, the class LinkedGraph could be for unweighted 
undirected graphs using linked adjacency lists and the class MatrixWDigraph 
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The lists are vertex0: +NO—Ni—N2 
vertex 1: NO->N3->N4 
vertex 2: NI —~N3—NS 
vertex 3: N2—N4—2N5 


Figure 6.12: Adjacency multilists for G, of Figure 6.2(a) 


could be for weighted directed graphs using a matrix representation. Since many 
of the functions (e.g., return the number of vertices and edges) we perform on 
graphs are independent of the graph type and representation, we can reduce the 
coding effort by defining a virtual class Graph (as in ADT 6.1) that is the reposi- 
tory for common codes. Classes such as LinkedGraph and MatrixWDigraph can 
then be derived (directly or indirectly) from Graph, thereby inheriting the com- 
mon functions. The classes for networks (i.e., graphs with weighted edges) may 
be template classes or we may restrict the edge weight to be of type double. Fig- 
ure 6.13 shows a possible derivation hierarchy for our graph classes. This figure 
includes only classes that use the matrix and linked adjacency list representa- 
tions. Classes for other representations may be added to the hierarchy. 

In the remaining sections of this chapter, we investigate several interesting 
problems defined on graphs. For convenience, this investigation, at times. 
assumes a definite representation for a graph. In many cases, the code for the 
corresponding function can be written in an implementation independent manner 
provided we have an iterator for each non-abstract class that derives from Graph. 
In these cases, the implementation-independent code may be physically placed 
in the abstract class Graph and used by all implementations. In fact, we may 
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Figure 6.13: Possible derivation hierarchy for graph classes 


additionally write customized codes for some of the implementations. For exam- 
ple, we may write a customized code for some function foo in the class Matrix- 
Graph and also have an implementation independent code for foo (using itera- 
tors) in the class Graph. On graphs of type MatrixGraph, the custom code is 
used while on graph types for which there is no custom code, the generic code in 
Graph is used. Custom implementation for foo, of course, would make sense 
only if the custom implementation has beneficial attributes (e.g., it runs faster) 
relative to the implementation independent code. 


EXERCISES 


|. Does the multigcaph of Figure 6.14 have an Eulerian walk? If so, find one. 


Figure 6.14: A multigraph 


The Graph Abstract Data Type 339 


2. For the digraph of Figure 6.15 obtain 


(a) 
(b) 
(c) 
(d) 
(e) 


the in-degree and out-degree of each vertex 
its adjacency-matrix 

its adjacency-list representation 

its adjacency-multilist representation 

its strongly connected components 


Figure 6.15: A digraph 


3. Draw the complete undirected graphs on one, two, three, four, and five ver- 
tices. Prove that the number of edges in an n-vertex complete graph is 
n(n —1)/2. 


4. Is the directed graph of Figure 6.16 strongly connected? List all the simple 


paths. 


Figure 6.16: A directed graph 
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Obtain the adjacency-matrix, adjacency-list, and adjacency-multilist 

Tepresentations of the graph of Figure 6.16. 

Show that the sum of the degrees of the vertices of an undirected graph is 

twice the number of edges. 

7, (a) Let Gbe a connected, undirected graph on a vertices, Show that G 

must have at least »—1 edges and that all connected, undirected 

graphs with n — 1 edges are trees, 

(b) What is the minimum number of edges in a strongly connected 
digraph on n vertices? What form do such digraphs have? 


For an undirected graph G with n vertices, prove that the following are 
equivalent: 


(a) Gisatree 


(b) Gis connected, but if any edge is removed the resulting graph is not 
connected 

(c) For any two distinct vertices ue V(G) and ve V(G), there is 
exactly one simple path trom u to v 


(a) G contains no cycles and has 1 — 1 edges 


9, Write a C++ function to input the number of vertices and edges in an 
undirected graph. Next, input the edges one by one and to set up the linked 
adjacency-list representation of the graph. You may assume that no edge is 
input twice. What is the run time of your function as a function of the 
number of vertices and the number of edges? 

10. Do the preceding exercise but this time set up the multilist representation. 


1]. Let G be an undirected, connected graph with at least one vertex of odd 
degree. Show that G contains no Eulerian walk. 

12. [Programming Project] Write C++ code for each of the classes in the 
hierarchy of Figure 6.13. Each class should derive publicly from its parent 
class in the hierarchy. You code for Graph should build upon that of ADT 
6.1. For weighted graphs, assume the edge weight is of type double. 


6.2 ELEMENTARY GRAPH OPERATIONS 


When we discussed binary trees in Chapter 5, we indicated that tree traversals 
were among the most frequently used tree operations. Thus, we defined and 
implemented preorder, inorder, postorder, and level-order tree traversals. An 
analogous situation occurs in the case of graphs. Given a graph G=(V, E) anda 
vertex rin V(G), we wish to visit all vertices in G that are reachable from v (i.e., 
all vertices that are connected to v). We shall look at two ways of doing this: 
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depth-first search and breadth-first search. Although these methods work on 
both directed and undirected graphs, the following discussion assumes that the 
graphs are undirected. 


6.2.1 Depth-First Search 


We begin by visiting the start vertex v. Next an unvisited vertex w adjacent to v 
is selected, and a depth-first search from w is initiated. When a vertex u is 
reached such that all its adjacent vertices have been visited, we back up to the 
last vertex visited that has an unvisited vertex w adjacent to it and initiate a 
depth-first search from w. The search terminates when no unvisited vertex can 
be reached from any of the visited vertices. This function is best described 
recursively as in Program 6.1. 


virtual void Graph ::DFS() // Driver 
{ 


visited = new bool [n }; 
4 visited is declared as a bool* data member of Graph 
fill (visited, visited + n, false); 
DFS (0); # start search at vertex 0 
delete [) visited; 


virtual void Graph::DFS(const int vy) // Workhorse 
{// Visit all previously unvisited vertices that are reachable from vertex v. 
visited [v ] = true; 
for (each vertex w adjacent to v) // actual code uses an iterator 
if (!visited [w]) DFS (w); 
} 


Program 6.1: Depth-first search 


Example 6.1: Consider the graph G of Figure 6.17(a), which is represented by 
its adjacency lists as in Figure 6.17(b). If a depth-first search is initiated from 
vertex 0, then the vertices of G are visited in the following order: 0, 1, 3, 7, 4, 5, 
2, 6. Since DFS(Q) visits ali vertices that can be reached from 0, the vertices 
visited, together with all edges in G incident to these vertices, form a connected 
component of G. 0 
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Figure 6.17: Graph G and its adjacency lists 


Analysis of DFS: When G is represented by its adjacency lists, the vertices w 
adjacent to v can be determined by following a chain of links. Since DFS exam- 
ines each node in the adjacency lists at most once, and there are 2e list nodes, the 
time to complete the search is O(e). If G is represented by its adjacency matrix, 
then the time to determine all vertices adjacent to v is O(n). Since at most n ver- 
tices are visited, the total time is O(n”), O 
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6.2.2. Breadth-First Search 


In a breadth-first search, we begin by visiting the start vertex v. Next, all 
unvisited vertices adjacent to v are visited. Unvisited vertices adjacent to these 
newly visited vertices are then visited, and so on. Algorithm BFS (Program 6.2) 
Sives the details. 


Example 6.2: Consider the graph of Figure 6.17(a). If a breadth-first search is 
performed beginning at vertex 0, then we first visit vertex 0. Next, vertices 1 and 
2 are visited. Then vertices 3, 4, 5, and 6, and finally 7, are visited. 0 


virtual void Graph::BFS(int v) 
{// A breadth first search of the gcaph is carried out beginning at vertex v. 
H visited [i] is set to true when v is visited. The function uses a queve. 
visited = new bool [n}; 
fill (visited, visited + n, false); 
visited [v ] = true; 
Queue<int> q; 
q-Push (v); 
while (!¢./sEmpty ()) { 
v= q_Front(); 
q.Pop (); 
for (all vertices w adjacent to v) // actual code uses an iterator 
if (‘visited [w}) { 
q.Push(w); 
visited [w] = true; 


} 
} 4 end of while loop 
delete [] visited; 


} 


Program 6.2: Breadth-first search 


Analysis of BFS: Each visited vertex enters the queue exactly once. So, the 
while loop is iterated at most # times. If an adjacency matrix is used, the loop 
takes O(n) time for each vertex visited. The total time is, therefore, O(n?). If 
adjacency lists are used, the loop has a total cost of dg + --- +d,-1 = Ofe), 
where d; is the degree of vertex i. As in the case of DFS, all visited vertices, 
together with all edges incident to them, form a connected component of G. 0 
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6.2.3 Connected Components 


If G is an undirected graph, then one can determine whether or not it is con- 
nected by simply making a call to either DFS or BFS and then determining if 
there is any unvisited vertex. The connected components of a graph may be 
obtained by making repeated calls to either DFS(v) or BFS(v), where v is a ver~ 
tex that has not yet been visited. This leads to function Components (Program 
6.3), which determines the connected components of G. The algorithm uses DFS 
(BFS may be used instead if desired). The computing time is not affected. Func- 
tion Graph::OutputNewComponent outputs all vertices visited in the most recent 
invocation of DFS, together with all edges incident on these vertices. 


virtual void Graph::Components () 
{/ Determine the connected components of the graph. 
4 visited is assumed to be declared as a bool* data member of Graph 
visited = new bool [11]; 
fill (visited, visited + n, false); 
for (i= 03 i <n; i++) 
if (visited [i}) { 
DFS (i); # find a component 
OutputNewComponent (); 


delete {] visited; 


} 


Program 6.3: Determining connected components 


Analysis of Components: If G is represented by its adjacency lists, then the 
total time taken by DFS is Ofe), The output can be completed in time Ofe) if 
DFS kecps a list of all newly visited vertices. Since the for loops take O(n) 
time, the total time to generate all the connected components is O(n +e). If 
adjacency matrices are used instead, the time required is O?). a 


6.2.4 Spanning Trees 


When the graph G is connected, a depth-first or breadth-first search starting at 
any vertex visits all the vertices in G. In this case the edges of G are partitioned 
into two sets, T (for tee edges) and N (for nontree edges), where T is the set of 
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edges used or traversed during the search and N the set of remaining edges. The 
set T may be determined by inserting the statement 7 = TU {(v,w)) in the if 
clauses of DFS and BFS. The edges in T form a tree that includes all the vertices 
of G. Any tree consisting solely of edges in G and including all vertices in G is 
called a spanning tree. Figure 6.18 shows a graph and some of its spanning 
trees. 


33K of 


Figure 6.18: A complete graph and three of its spanning trees 


As indicated earlier, a spanning tree may be constructed using either a 
depth-first or a breadth-first search. The spanning tree resulting from a depth- 
first search is known as a depth-first spanning tree. When a breadth-first search 
is used, the spanning tree is called a breadth-first spanning tree. Figure 6.19 
shows the spanning trees resulting from a depth-first and breadth-first search 
Starting at vertex 0 of the graph of Figure 6.17. 


(a) DFS (0) spanning tree (b) BFS (0) spanning tree 


Figure 6.19: Depth-first and breadth-first spanning trees for graph of Figure 6.17 


If a nontree edge (v, w) is introduced into any spanning tree T, then a cycle 
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is formed, This cycle consists of the edge (v,w) and all the edges on the path 
from w lovin T. For example, if the edge (7,6) is introduced into the DFS span- 
ning tree of Figure 6.19(a), then the resulting cycle is 7,6,2,5,7. We can use this 


property of spanning trees to obtain an independent set of circuit equations for an 
electrical network. 


Example 6.3 [Creation of circuit equations): To obtain the circuit equations, we 
must first obtain a spanning tree for the electrical network. Then we introduce 
the nontree edges into the spanning tree one at a time. The introduction of each 
such edge produces acycle. Next we use Kirchhoff's second law on this cycle to 
obtain a circuit equation. The cycles obtained in this way are independent (we 
cannot obtain any of these cycles by taking a linear combination of the remain- 
ing cycles), since each contains a nontree edge that is not contained in any other 
cycle. Thus, the circuit equations are also independent. In fact, we can show 
that the cycles obtained by introducing the nontree edges one at a time into the 
spanning tree form a cycle basis. This means that we can construct all other 
cycles in the graph by taking a linear combination of the cycles in the basis.O 


Let us examine a second property of spanning trees. A spanning tree is a 
minimal subgraph, G’, of G such that V(G) = V(G), and G’ is connected. We 
define a minimal subgraph as one with the fewest number of edges. Any con- 
nected graph with n vertices must have at least n — 1 edges, and all connected 
graphs with n — | edges are trees. Therefore, we conclude that a spanning tree 
has n — | edges. (The Exercises explore this property more fully.) 

Constructing minimal subgraphs finds frequent application in the design of 
communication networks. Suppose that the vertices of a graph G represent 
cities, and the edges represent communication links between cities. The 
minimum number of links needed to connect n cities is n - 1, Constructing the 
spanning trees of G produces all feasible choices. However, we know that the 
cost of constructing communication links between cities is rarely the same. 
Therefore, in practical applications, we assign weights to the edges. These 
weights might represent the cost of constructing the communication fink or the 
length of the link. Given such a weighted graph, we would like to select the 
spanning wee that represents either the lowest total cost or the lowest overall 
length. We assume that the cost of a spanning tree is the sum of the costs of the 
edges of that tree. Algorithms to obtain minimum-cost spanning trees are studied 
in Section 6.3. 
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6.2.5 Biconnected Components 


The operations that we have implemented thus far are simple extensions of 
depth-first and breadth-first searches. The next operation we implement is more 
complex and requires the introduction of additional terminology. We begin by 
assuming that G is an undirected, connected graph. 


Definition: A vertex v of G is an articulation point iff the deletion of v, together 
with the deletion of all edges incident to v, leaves behind a graph that has at least 
two connected components. 0 


Vertices 1, 3, 5, and 7 are the articulation points of the connected graph of 
Figure 6.20(a). 


Definition: A biconnected graph is a connected graph that has no articulation 
points. O 


The graph of Figure 6.20 is not biconnected, and that of Figure 6.17(a) is. 
Articulation points are undesirable in graphs that represent communication net- 
works. In such graphs the vertices represent communication stations, and the 
edges represent communication links. The failure of a communication station 
that is an articulation point results in a loss of communication to stations other 
than the one that failed. If the communication graph is biconnected, then the 
failure of a single station results in a loss of communication to and from only that 
Station, 


Definition: A biconnected component of a connected graph G is a maximal 
biconnected subgraph H of G. By maximal, we mean that G contains no other 
subgraph that is both biconnected and properly contains H. O 


The graph of Figure 6.20(a) contains six biconnected components. These 
are shown in Figure 6.20(b). Note that a biconnected graph has just one bicon- 
nected component: the whole graph. It is easy to verify that two biconnected 
components of the same graph can have at most one vertex in common, From 
this it follows that no edge can be in two or more biconnected components. 
Hence, the biconnected components of G partition the edges of G. 

The biconnected components of a connected, undirected graph G can be 
found by using any depth-first spanning tree of G. For the graph of Figure 
6.20(a) a depth-first spanning tree with root 3 is shown in Figure 6.21(a). This 
tree is redrawn in Figure 6.21(b) to better reveal the tree structure. This figure 
also shows the nontree edges of G by broken lines. The numbers outside the ver- 
tices give the sequence in which the vertices are visited during the depth-first 
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Figure 6.20: A connected graph and its biconnected components 


search. This number is called the depth-first number, dfn, of the vertex. So, 
dfn (0) = 5, and dfn (9) = 9. Note that if u and v are two vertices such that «# is an 
ancestor of v in the depth-first spanning tree, then dfn (u) < dfn(v). 

The broken lines in Figure 6.21(b) represent nontree edges. A nontree edge 
{u, v) is a back edge with respect to a spanning tree T iffeither u is an ancestor of 
vor vis an ancestor of x. A nontree edge that is not a back edge is called a cross 
edge. The nontree edges (3, 1) and (5, 7) are back edges. From the definition of 
a depth-first search, one can show that no graph can have cross edges with 
respect to any of its depth-first spanning trees. From this, it follows that the root 
of the depth-first spanning tree is an articulation point iff it has at least two chil- 
dren. Further, any other vertex u is an articulation point iff it has at least one 
child, , such that it is not possible to reach an ancestor of w using a path 
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Figure 6.21: Depth-first spanning tree of Figure 6.20(a) 


composed solely of w, descendants of w, and a single back edge. These observa- 
tions lead us to define a value low for each vertex of G such that /ow(w) is the 
lowest depth-first number that can be reached from w using a path of descendants 
followed by, at most, one back edge. low (w) is given by the equation 


low (w) = min(dfn (w), min (low (x) | x is a child of w), 
min {dfn (x) | (w, x) is a back edge}} 


From the preceding discussion it follows that u is an articulation point iff « 
is either the root of the spanning tree and has two or more children or w is not the 
root and « has a child w such that low(w) 2 dfn(u). Figure 6.22 gives the dfn 
and low values for each vertex of the spanning tree of Figure 6.21 (b). 


vertex | O] 1 | 2 [3] 4[5]6|7 8] 9] 
dfn s[4 3/1{[2[6 ae 9| 
low S{[i;itlifvirypele le] wo] 9] 


Figure 6.22: dfn and low values for the spanning tree of Figure 6.21(b) 


Function DFS is easily modified to compute dfn and /ow for each vertex of 
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a connected graph, The Tesult is function DfnLow (Program 6.4). This function 
uses the function min, which returns the smaller of its two parameters. 


ta void Graph::DfnLow(const int x) // begin DFS at vertex x 


mun = 1; . 4 nwn is an int data member of Graph 
dfn = new int(n}; dfn is declared as int* in Graph 

Jow = new int[n]; flow is declared as int* in Graph 

Sill (dfn, dfn +n, 0); 
fill(low, low +n, 0); 
DfnLow (x,-1)3 i start at vertex x 
delete [] dfn; 

delete [) low; 


} 


void Graph::DfuLow (const int «, const int v) 
st Compute dfnr and fow while performing a depth first search beginning at vertex 
Mu. v is the parent (if any) of w in the resulting spanning tree. 
dfn (u} = low [u] = num++; 
for (each vertex sv adjacent from u) // actual code uses an iterator 
if (dfn (w} ==0) { / wis an unvisited vertex 
DfnLow (w,u); 
low [1] = min (low [u}, low [w))3 


} 
; else if (w != v) low [u] = min (low (u}, dfn [w]); back edge 


Program 6.4: Computing dfn and low 


The edges of the connected graph may be partitioned into their biconnected 
components by adding some code to function DfnLow. First, note that following 
the return from DfnLow (ww, u), low [w] has been computed. If low [w] 2 dfn (u), 
then a new biconnected component has been identified. By using a stack to save 
edges when they are first encountered, we can output all edges in a biconnected 
component, as in function Biconnected (Program 6.5). 

Establishing the correctness of function Biconnected is left as an exercise. 
Its complexity is O(# +e). Note that function Biconnected assumes that the input 
connected graph has at least two vertices. Connected graphs with just one vertex 
contain no edges. By convention these graphs are biconnected, and a proper 
biconnected components function should handle them as a special case, produc- 
ing a single biconnected component as output. 
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virtual void Graph::Biconnected () 


} 


num = 15 4 num is an int data member of Graph 
dfn = new int[n]; H dfn is declared as int* in Graph 
low = new int[n}; H low is declared as int* in Graph 
fill (dfn, dfn + n, 0) 

fill(low, low + n, 0); 

Biconnected (0,—1); // start at vertex 0 

delete [] dfn; 

delete {] low; 


virtual void Graph::Biconnected (const int u, const int v) 

{// Compute dfn and low, and output the edges of G by their biconnected 
4 components. v is the parent (if any) of u in the resulting spanning tree. 
4s is an initially empty stack declared as a data member of Graph. 


} 


dfn{u)= low[u]= num+4; 
for ( each vertex w adjacent from x) { // actual code uses an iterator 
if ((v != w) && (din{w] < dfn [u])) add (u, w) to stack s; 
if (dfn [w ] = 0) { // w is an unvisited vertex 
Biconnected (w,u); 
low [u] = min (low [u), low{w)); 
if (low [w] >= dfn(u}) { 
ea << "New Biconnected Component: " << endl; 
do 
delete an edge from the stack s; 
let this edge be (x, y); 
cout << x <<", "<< y << endl; 
} while ( (x, y) and (u, w) are not the same edge) 


} 
else if (w != v) low [u]= min (low [u], din [w)); // back edge 


Program 6.5: Outputting biconnected components when n> 1 
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EXERCISES 


1. 


Apply depth-first and breadth-first searches to the complete graph on four 
vertices. List the vertices in the order they would be visited. 


Write a complete C++ function for depth-first search under the assumption 
that graphs are represented using adjacency matrices. Test the correctness 
of your function using suitable graphs. 

Write a complete C++ function for depth-first search under the assumption 
that graphs are represented using adjacency lists. Test the correctness of 
your function using suitable graphs. 

Write a complete C++ function for breadth-first search under the assump- 
tion that graphs are represented using adjacency matrices. Test the correct- 
ness of your function using suitable graphs. 

Write a complete C++ function for breadth-first search under the assump- 
tion that graphs are represented using adjacency lists. Test the correctness 
of your function using suitable graphs. 

Show how to modify function DFS (Program 6.1), as it is used in Com- 
ponents (Program 6.3), to produce a list of all newly visited vertices. 

Prove that when function DFS (Program 6.1) is applied to a connected 
graph, the edges of T form a tree. 

Prove that when function BFS (Program 6.2) is applied to a connected 
graph, the edges of T forma tree. 

Show that if 7 is a spanning tree for the undirected graph G, then the addi- 
tion of an edge e, e € E(T) and e € E(G), to T creates a unique cycle, 

Show that the number of spanning trees in a complete graph with 1 vertices 
is at least 2"-! - 1. 

Let G be a connected graph and let 7 be any of its depth-first spanning 
trees. Show that G has no cross edges relative to T. 

Prove that function Biconnected (Program 6.5) correctly partitions the 
edges of a connected graph into the biconnected components of the graph. 
Let G be a connected, undirected graph. Show that no edge of G can be in 
two or more biconnected components of G. 


6.3 MINIMUM-COST SPANNING TREES 


The cost of a spanning tree of a weighted, undirected graph is the sum of the 
costs (weights) of the edges in the spanning tree. A minimun-cost spanning tree 
is a spanning tree of least cost. Three different algorithms can be used to obtain 
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a minimum-cost spanning tree of a connected, undirected graph. Ail three use a 
design strategy called the greedy method. We shall refer to the three algorithms 
as Kruskal’s, Prim’s, and Sollin’s algorithms, respectively. 

In the greedy method, we constrict an optimal solution in stages. At each 
stage, we make the best decision (using some criterion) possible at the time. 
Since we cannot change this decision later, we make sure that the decision will 
result in a feasible solution. The greedy method can be applied to a wide variety 
of programming problems. Typically, the selection of an item at each stage is 
based on either a least-cost or a highest profit criterion. A feasible solution is 
one that works within the constraints specified by the problem. 

To construct minimum-cost spanning trees, we use a least-cost criterion. 
Our solution must satisfy the following constraints: 


{1) We must use only edges within the graph. 
(2) We must use exactly n — | edges. 
(3) We may not use edges that produce a cycle. 


6.3.1 Kruskal’s Algorithm 


Kruskal’s algorithm builds a minimum-cost spanning tree T by adding edges to T 
one at atime. The algorithm selects the edges for inclusion in T in nondecreas- 
ing order of their cost. An edge is added to T if it does not form a cycle with the 
edges that are already in 7. Since G is connected and has n > 0 vertices, exactly 
n—1 edges will be selected for inclusion in 7. 


Example 6.4: We will construct a minimum-cost spanning tree of the graph of 
Figure 6.23(a). We begin with no edges selected. Figure 6.23(b) shows the 
current graph with no edges selected. Edge (0,5) is the first edge considered. It 
is included in the spanning tree being built. This yields the graph of Figure 
6.23(¢). 

Next, the edge (2,3) is selected and included in the tree (Figure 6.23(d)). 
The next edge to be considered is (1,6). Its inclusion in the tree being built does 
fot create a cycle, so we get the graph of Figure 6.23(e). Edge (1.2) is con- 
sidered next and included in the tree (Figure 6.23(f)). Of the edges not yet con- 
sidered, (6,3) has the least cost. It is considered next. Its inclusion in the tree 
results in a cycle, so this edge is discarded. Edge (4,3) is the next edge to be 
added to the tree being built. This results in the configuration of Figure 6.23(g). 
The next edge to be considered is the edge (6,4). It is discarded, as its inclusion 
creates a cycle. Finally, edge (5,4) is considered and included in the tree being 
built. AS completes the spanning wee. The resulting tree (Figure 6.23(h)) has 
cost 99. O 
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Figure 6.23: Stages in Kruskal’s algorithm 
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It is somewhat surprising that this straightforward approach should always 
result in a minimum-cost spanning tree. We shall soon prove, however, that this 
is indeed the case. First, let us look into the details of the algorithm. For clarity, 
Knuskal’s algorithm is written out more formally in Program 6.6. Initially, E is 
the set of all edges in G. The only functions we wish to perform on this set are 


(1) determine an edge with minimum cost (line 3) 
(2) delete this edge (line 4) 


Both these functions can be performed efficiently if the edges in E are main- 
tained as a sorted sequential list. In Chapter 7 we shall see how to sort these 
edges into nondecreasing order in time Ofe loge), where e is the number of 
edges in E. It is not essential to sort all the edges, as long as the next edge for 
line 3 can be determined easily. This is an instance where a min heap is ideal, as 
it permits the next edge to be determined and deleted in O(log e) time. The con- 
struction of the heap itself takes Of) time. 


1 T=, 

2 while ((T contains less than n — 1 edges) && (E not empty)) { 
3 choose an edge (v,w) from E of lowest cost; 

4 delete (v,w) from E; 

5 if ((v,w) does not create a cycle in T) add (v,w) to T; 

6 else discard (v,w); 

7} 

8 


if (T contains fewer than n — 1 edges) cout << "no spanning tree” << endl; 


Program 6.6: Kruskal’s algorithm 


To perform line 5 of Program 6.6 efficiently, the vertices in G should be 
grouped together in such a way that one may easily determine if the vertices v 
and w are already connected by the earlier selection of edges. If they are, then 
the edge (v,w) is to be discarded. If they are not, then (v,w) is to be added to T. 
One possible grouping is to place all vertices in the same connected component 
of T into a set (all connected components of T will also be trees). Then, two ver- 
tices v and w are connected in 7 iff they are in the same set. For example, when 
the edge (3, 6) is to be considered, the sets would be {0, 5}, [1, 2,3, 6}, and [4}. 
Vertices 3 and 6 are already in the same set, so the edge (3, 6) is rejected. The 
next edge to be considered is (3, 4). Since vertices 3 and 4 are in different sets, 
the edge is accepted. This edge connects the two components {1, 2, 3, 6} and 
{4}, so these two sets should be unioned to obtain the set representing the new 
component. Using the set representation scheme of Chapter 5, we can obtain an 
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efficient implementation of line 5. The computing time is, therefore, determined 
by the time for lines 3 and 4, which in the worst case is Ofe log e). We leave the 
writing of the resulting algorithm as an exercise. Theorem 6.1 proves that the 


eerie resulting from Program 6.6 does yield a minimum-cost spanning tree 
of G. 


Theorem 6.1: Let G be any undirected, connected graph. Kruskal’s algorithm 
generates a minimum-cost spanning tree. 


Proof: We shall show the following: (a) Kruskal’s method results in a spanning, 
tee whenever a spanning tree exists; and (b) the spanning tree generated is of 
minimum cost, 

For (a), we note that the only edges that get discarded in Kruskal's method 
are those that result in a cycle. The deletion of a single edge that is on a cycle of 
connected graph results in a graph that is also connected. Hence, if G initially 
is connected, the set of edges in Tand E always forma connected graph. Conse- 
quently, if G initially is connected, the algorithm cannot terminate with E = @ 
and ITI <n-1, 

___ Now, let us proceed to establish that the constructed spanning tree, T, is of 
minimum cost. Since G has a finite number of spanning trees, it must have at 
least one of minimum cost. Let U be a minimum-cost spanning tree. Both 7 and 
U have exactly n-1 edges. If T = U, then T is of minimum cost, and we have 
nothing to prove, So, assume that T # U. Let k, k > 0, be the number of edges in 
T that are not in U. Note that k is also the number of edges in U that are not in T. 

We shall show that T and U have the same cost by transforming U into 7. 
This transformation will be done in k steps. At each step, the number of edges in 
T that are not in U will be reduced by exactly 1, Further, the cost of U will not 
change as a result of the transformation. As a result, U after k steps of transfor- 
mation will have the same cost as the initial U/ and will consist of exactly those 
edges that are in T. This implies that 7 is of minimum cost. 

Each step of the transformation involves adding to U one edge, e, from T 
and removing one edge, f, from U. The edges ¢ and-fare selected in the follow- 
ing way: 


(1) Lete be the least-cost edge in T that is not in U. Such an edge must exist 
ask > 0. 


(2) When ¢ is added to U, a unique cycle is created. Let fbe any edge on this 
cycle that is not in 7: Note that at least one of the edges on this cycle is not 
in 7, as T contains no cycles. 


From the way e and f are selected, it follows that V= U + {e) — {ffisa 
spanning tree and that T has exactly k—-1 edges that are not in V. We need to 
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show that the cost of V is the same as that of U. Clearly, the cost of V is the cost 
of U plus the cost of the edge e minus the cost of the edge f. The cost of e cannot 
be less than the cost of f, as otherwise the spanning tree V has a smaller cost than 
the tree U, which is impossible. If ¢ has a higher cost than f, then fis considered 
before e by Kruskal’s algorithm. Since f is not in 7, Kruskal’s algorithm must 
have discarded this edge at this time. Hence, f, together with edges in T having a 
cost less than or equal to the cost of f, must forma cycle. By the choice of ¢, all 
these edges are also in U. Hence, U must also contain a cycle. But it does not, 
as it is a spanning tree. So, the assumption that ¢ is of higher cost than fleads to 
a contradiction. The only possibility that remains is that e and f have the same 
cost. Hence, Vhas the same cost as U. 0 


6.3.2 Prim’s Algorithm 


Prim’s algorithm, like Kruskal’s, constructs the minimum-cost spanning tree edge 
by edge. However, at all times during the algorithm, the set of selected edges 
forms a tree. (By contrast, the set of selected edges in Kruskal’s algorithm forms 
a forest at all times.) Prim’s algorithm begins with a tree T that contains a single 
vertex. This vertex can be any of the vertices in the original graph. Then we add 
a least-cost edge (u, v) to T such that TU {(u, v)} is also a tree. This edge- 
addition step is repeated until T contains n—1 edges. Notice that edge (u, v) is 
always such that exactly one of u and y is in 7. A high-level description of 
Prim’s algorithm is provided in Program 6.7. This description also provides for 
the possibility that the input graph may not be connected. In this case there is no 
spanning tree. Figure 6.24 shows the progress of Prim's algorithm on the graph 
of Figure 6.23(a). 


# Assume that G has at least one vertex. 

TV = (0}; / start with vertex 0 and no edges 

for (T = ©; T contains fewer than n—1 edges; add (u, v) to 7) 

{ 
Let (u, v) be a Jeast-cost edge such that ue TV and v ¢ TV; 
if (there is no such edge) break; 
add v to TV; 


if (J contains fewer than n—1 edges) cout << “no spanning tree” << endl; 


Program 6.7: Prim’s algorithm 


Prim’s algorithm can be implemented to have a time complexity O(n") if 
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@) 


Figure 6.24: Stages in Prim’s algorithm 


we associate with each vertex v not in TV a vertex near (v) such that near(v) € 
TV and cost (near(v), v) is the minimum over al) such choices for near (v) (we 
assume that cast (v, w) = if (¥, w)¢€ £). The next edge to add to T is such that 
cost (near (v), v) is the minimum and v € TV. Asymptotically faster implementa- 
tions are also possible. One of these results from the use of Fibonacci heaps, 
which are studied in Chapter 9. Establishing the correctness of Prim’s algorithm 
is left as an exercise. 
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6.3.3 Sollin’s Algorithm 


Sollin’s algorithm selects several edges at each stage. At the start of a stage, the 
selected edges, together with all n graph vertices, form a spanning forest. During 
a stage we select one edge for each tree in this forest. This edge is a minimum- 
cost edge that has exactly one vertex in the tree. The selected edges are added to 
the spanning tree being constructed. Note that it is possible for two trees in the 
forest to select the same edge. So, multiple copies of the same edge are to be 
eliminated. Also, when the graph has several edges with the same cost, it is pos- 
sible for two tees to select two different edges that connect them together. 
These edges will, of course, have the same cost; only one of these should be 
Tetained. At the start of the first stage, the set of selected edges is empty. The 
algorithm terminates when there is only one tree at the end of a stage or when no 
edges remain to be selected. 

Figure 6.25 shows the stages in Sollin’s algorithm when it begins with the 
graph of Figure 6.23(a). The initial configuration of zero selected edges is the 
same as that shown in Figure 6.23(b). Each tree in this spanning forest is a sin- 
gle vertex. The edges selected by vertices 0, 1, ---, 6 are, respectively, (0, 5), 
(1, 6), (2, 3), (3, 2), (4, 3), (5, 0), and (6, 1). The distinct edges in this selection 
are (0, 5), (1, 6), (2, 3), and (4, 3). Adding these to the set of selected edges 
results in the configuration of Figure 6.25(a). In the next stage, the tree with ver- 
tex set {0,5} selects the edge (5, 4), and the remaining two trees select the edge 
(1, 2). Following the addition of these two edges to the set of selected edges, 
construction of the spanning tree is complete. The resulting spanning tree is 
shown in Figure 6.25(b). The development of Sollin’s algorithm into a C++ 
function and its correctness proof are left as exercises. 


EXERCISES 


1, Write out Kruskal’s minimum-cost spanning tree algorithm (Program 6.6) 
as a complete program. You may use as functions the algorithms 
WeightedUnion (Program 5.22) and CollapsingFind (Program 5.23). Use a 
min-heap (Chapter 5) to select the edges in nondecreasing order by weight. 

2. Prove that Prim’s algorithm finds a minimum-cost spanning tree for every 
connected, undirected graph. 

3. Refine Program 6.7 into a C++ function to find a minimum-cost spanning 
tree. The complexity of your function should be O(n), where n is the 
number of vertices in the input graph. Show that this is the case. 

4. Prove that Sollin’s algorithm finds a minimum-cost spanning tree for every 
connected, undirected graph. 
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Figure 6.25: Stages in Sollin's algorithm 


5. What is the maximum number of stages in Sollin’s algorithm? Give this as 


a function of the number of vertices n in the graph. 


6, Obtain a C++ function to find a minimum-cost spanning tree using Sollin’s 
algorithm. What is the complexity of your function? 


6.4 SHORTEST PATHS AND TRANSITIVE CLOSURE 


Graphs may be used to represent the highway structure of a state or country with 
vertices representing cities and edges representing sections of highway. The 
edges may then be assigned weights, which might be the distance between the 
two cities connected by the edge or the average time to drive along that section 
of highway. A motorist wishing to drive from city A to city B would be 
interested in answers to the following questions: 


(1) Is there a path from A to B? 
(2) If there is more than one path from A to B, which is the shortest path? 


The problems defined by (1) and (2) above are special cases of the path problems 
we shall be studying in this section. An edge weight is also referred to as an 
edge length or edge cost. We shall use the terms weight, cost, and length inter- 
changeably. The length (cost, weight) of a path is now defined to be the sum of 
the lengths (costs, weights) of the edges on that path, rather than the number of 
edges. The starting vertex of the path will be referred to as the source and the 
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last vertex the destination. The graphs will be digraphs to allow for one-way 
streets. 


6.4.1 Single Source/All Destinations: Nonnegative Edge Costs 


In this problem we are given a directed graph G =(V,E), a length function 
length (i,j), length (i,j) 2 0, for the edges of G, and a source vertex v. The prob- 
lem is to determine a shortest path from v to each of the remaining vertices of G, 
As an example, consider the directed graph of Figure 6.26(a). The numbers on 
the edges are the edge fengths. If vertex 0 is the source vertex, then the shortest 
path from 0 to 1 is 0, 3, 4, 1. The length of this path is 10 + 15 +20 = 45, Even 
though there are three edges on this path, it is shorter thar the path 0, 1, which is 
of length 50. There is no path from 0 to 5. Figure 6.26(b) lists the shortest paths 
from 0 to 1, 2, 3, and 4. The paths have been listed in nondecreasing order of 
path length. A greedy algorithm will generate the shortest paths in this order, 


Path Length 
1) 0,3 10 
2) 0,3,4 25 
3) 0,3,4, 1 45 
4) 0,2 45 
(a) Graph (b) Shortest paths from 0 


Figure 6.26: Graph and shortest paths from vertex 0 to all destinations 


Let S denote the set of vertices (including the source v) to which the shor- 
test paths have already been found. For w not in S, let dist {w] be the length of 
the shortest path starting from v, going through only the vertices that are in S, 
and ending at w. We observe that when paths are generated in nondecreasing 
order of length, 


(1) If the next shortest path is to vertex u, then the path begins at v, ends at 
4, and goes through only vertices that are in S. To prove this we must show that 
all of the intermediate vertices on the shortest path to u must be in S. Assume 
there is a vertex w on this path that is not in S. Then, the v-to-u path also con- 
tains a path from v to w that is of length less than that of the v-to-w path. By 
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assumption, the shortest paths are being generated in nondecreasing order of 
path length, So the shorter path from v to w has been generated already. Hence, 
there is no intermediate vertex that is not in S. 


(2) The destination of the next path generated must be the vertex that has 
the minimum distance, dist [1], among all vertices not in S. This follows from 
the definition of dist and observation (1). If there are several vertices not in S 
with the same dist, then any of these may be selected. 


(3) The vertex u selected in (2) becomes a member of 5. The shortest v-to- 
4 path is obtained from the selection process of (2). At this point, the length of 
the shortest paths starting at v, going through vertices only in S, and ending at a 
vertex w’ not in S may decrease (i.e., the value of dist [w] may change). If it does 
change, then the change must be due to a shorter path starting at v going to u and 
then (ow. The intermediate vertices on the v-to-u path and the u to w path must 
all be in S. Further, the y-to-w path must be the shortest such path; otherwise 
dist [w] is not defined properly. Also, the 1-to-w path can be chosen so that it 
does not contain any intermediate vertices. Therefore, we may conclude that if 
dist |w ] changes (i.e., decreases), then the change is due to the path from v to u to 
w, where the path from v to wis the shortest such path and the path from u to w is 
the edge <u,w>, The length of this path is dist [w] + length (<u,w>). 


‘The function ShortestPath (Program 6.8) uses these observations to deter- 
mine the length of the shortest path from v to each of the other vertices in G. The 
generation of the paths is a minor extension of the algorithm and is left as an 
exercise. It is assumed that the n vertices of G are numbered 0 through n-1. 
The set S is maintained as a Boolean array with s[i} = false if vertex i is notin S 
and s[i] = true if it is. It is assumed that the graph itself is represented by its 
length-adjacency matrix, with length [i JL] being the length of the edge <i,j>. If 
<i, /> is not an edge of the graph and i # j, length[i J[/] may be set to some large 
number LARGE. The choice of this number is arbitrary, although we make two 
stipulations regarding its value: 


(1) The number must be larger than any of the values in the length matrix. 


{2} The number must be chosen so that the statement dist{u] + length[u][ w) 
does not produce an overflow. 

Restriction (2) makes MAXDOUBLE (in case edge weights are of type double) a 
poor choice for nonexistent edges. For ==), length [i }L/] may be set to any non- 
negative number without affecting the outcome of the function. The arrays 
length, dist and s as well as the function Choose used by Program 6.8 are 
assumed to be defined elsewhere in the class MatrixWDigraph and accessible 
from Program 6.8. 
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1 void MatrixWDigraph::ShortestPath(const int n, const int v) 

2 (/ dist {j), 0 <j <n, is set to the length of the shortest path from v to j 

3 //in a digraph G with n vertices and edge lengths given by length [i ][/). 

4 for (int i = 0; i <n; i++) {s [i] = false; dist [7} = length (v }[i]3} # initialize 
5 s{vj]=true; 


6 dist[v]=0; 
7 for (i = 0; i<n—2; i++) { / determine n—) paths from vertex v 
8 int u = Choose (n); / choose returns a value u such that: 
9 Hf dist(u) = minimum dist [w), where s[w ] = false 
10 s[u) = true; 
0] for (int w = 0; w <n; w++) 
12 if (! s[w) && dist [u} + length [u){w] < dist[w]) 
13 dist (w ] = dist [u] + length [u Jw); 
14 } Mend of for (i=0;...) 
15} 


Program 6.8: Determining the lengths of the shortest paths 


Analysis of ShortestPath: From our earlier discussion, it is easy to see that the 
algorithm works. The time taken by the algorithm on a graph with n vertices is 
O(n?). To see this, note that the for loop of line 4 takes O(n) time. The for loop 
of line 7 is executed » — 2 times. Each execution of this loop requires O(n) time 
at line 8 to select the next vertex and again in lines 11 to 13 to update dist. So, 
the total time for this loop is O(n). If a list T of vertices currently not in S is 
maintained, then the number of nodes on this list would at any time be n-i. 
This, would speed up lines 8 and 11 to 13, but the asymptotic time would remain 
O(n?). This and other variations of the algorithm are explored i in the exercises. 
Any shortest-path algorithm must examine each edge in the graph at least 
once, since any of the edges could be in a shortest path. Hence, the minimum 
possible time for such an algorithm would be Oe). pbince length-adjacency 
matrices were used to represent the graph, it takes O(n?) time just to determine 
which edges are in G, so any shortest-path algorithm, that uses this representa- 
tion must take O(n"). For this representation, then, algorithm ShortestPath is 
optimal to within a constant factor. Even if a change to adjacency lists is made, 
only the overall time for the for loop of lines 11 to 13 can be brought down to 
Ofe) (since the dist can change only for vertices that are adjacent from u). The 
total time for line 8 remains O(n*). By using Fibonacci heaps (see Chapter 9) 
and adjacency lists, the greedy algorithm for the single-source/all-destinations 
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problem can be implemented to have complexity O(nlogn +e). For sparse 
graphs, this implementation is superior to that of Program 6.8. 


Example 6.5: Consider the eight-vertex digraph of Figure 6.27(a) with length- 
adjacency matrix as in Figure 6.27(b). Suppose that the source vertex is Boston. 
The values of dist and the vertex u selected in each iteration of the for loop of 
lines 7 to 14 (Program 6.8) are shown in Figure 6.28. We use ce to denote the 
value LARGE. Note that the algorithm terminates after only 6 iterations of the 
for loop. By the definition of dist, the distance of the last vertex, in this case Los 
Angeles, is correct, as the shortest path from Boston to Los Angeles can go 
through only the remaining six vertices. O 


6.4.2 Single Source/All Destinations: General Weights 


We now consider the general case when some or all of the edges of the directed 
graph G may have negative length. To see that function ShortestPath (Program 
6.8) does not necessarily give the correct results on such graphs, consider the 
graph of Figure 6,29. Let v = 0 be the source vertex. Since n = 3, the loop of 
lines 7 to 14 is iterated just once; u = 2 in line 8, and no changes are made to 
dist. The function terminates with dist (1) = 7 and dist [2] =5. The shortest path 
from 0 to 2 is 0, 1, 2. This path has length 2, which is less than the computed 
value of dist (2). 

When negative edge lengths are permitted, we require that the graph have 
no cycles of negative length. This is necessary so as to ensure that shortest paths 
consist of a finite number of edges. For example, consider the graph of Figure 
6.30. The length of the shortest path from vertex 0 to vertex 2 is —co, as the 
length of the path 


0, 1,0, 1,0, 2, -++,0,1,2 


can be made arbitrarily small. This is so because of the presence of the cycle 0, 
1,0, which has a length of -1. 

When there are no cycles of negative length, there is a shortest path 
between any two vertices of an n-vertex graph that has at most n — | edges on it. 
To see this, observe that a path that has more than n— | edges must repeat at least 
one vertex and hence must contain a cycle. Elimination of the cycles from the 
path results in another path with the same source and destination. This path is 
cycle-free and has a length that is no more than that of the original path, as the 
length of the eliminated cycles was at least zero. We can use this observation on 
the maximum number of edges on a cycle-free shortest path to obtain an algo- 
rithm to determine a shortest path from a source vertex to all remaining vertices 
in the graph. As in the case of function ShortestPath (Program 6.8), we shall 
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{b) Length-adjacency matrix 


Figure 6.27: Digraph for Example 6.5 


compute only the length, dist {w}, of the shortest path from the source vertex v to 
u. An exercise examines the extension needed to construct the shortest paths. 
Let dist'[u] be the length of a shortest path from the source vertex v to ver- 
tex i under the constraint that the shortest path contains at most / edges. Then, 
dist \(u] = length {v)[u], OSu <n. As noted earlier, when there are no cycles of 
negative length, we can limit our search for shortest paths to paths with at most 
— 1 edges. Hence, dist”~'[«] is the length of an unrestricted shortest path from 
v io uw 


Our goal then is to compute dist”~!{u] for all x. This can be done using 
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Figure 6.28: Action of ShortestPath on digraph of Figure 6.27 


5. 


Figure 6.29: Directed graph with a negative-length edge 


Figure 6.30: Directed graph with a cycle of negative length 


the dynamic programming methodology. First, we make the following observa- 
tions: 


(1) If the shortest path from v to w with at most k, k > 1, edges has no more than 
k—Ledges, then dist*[u | = dist*'{u }. 

(2) If the shortest path from v to « with at most k, k > 1, edges has exactly k 
edges, then it is comprised of a shortest path from v to some vertex j 
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followed by the edge <j,u>. The path from v to j has k~1 edges, and its 
length is dist*~'[j]. All vertices i such that the edge <i,u> is in the graph 
are candidates for j. Since we are interested in a shortest path, the i that 
minimizes dist*'{i] + length [i][u] is the correct value for j. 


These observations result in the following recurrence for dist: 


dist*(u] = min{dist*“'[u], min (dist! (i } + length{i)[u}}} 


This recurrence may be used to compute dist* from dist*~', for k = 2, 3, 
seraHl, 


Example 6.6: Figure 6.31 gives a seven-vertex graph, together with the arrays 
dist*,k =\, -++,6. These arrays were computed using the equation just given, 
o 


(a) A directed graph (b) dise* 


Figure 6.31: Shortest paths with negative edge lengths 


An exercise shows that if we use the same memory location dist (uw) for 
dist*{u), k = 1, -**, n—1, then the final value of dist{w) is still dist””!{u]. 
Using this fact and the recurrence for dist shown above, we arrive at the algo- 
rithm of Program 6.9 to compute the length of the shortest path from vertex v to 
each other vertex of the graph. This algorithm is referred to as the Beilman and 
Ford algorithm. 

Analysis of BellmanFord: Each iteration of the for loop of lines 4 to 7 takes 
O(n?) time if adjacency matrices are used and O(e) time if adjacency lists are 
used. The overall complexity is O(n7) when adjacency matrices are used and 


368 Graphs 


1 void MatrixWDigraph::BellmanFord(const int n, const int v) 
2 {# Single source all destination shortest paths with negative edge lengths. 
3 for (int § =0; i <n; i++) dist [i] = length [v [i]; / initialize dist 


4 for (int k= 2; k <=n-1; k++) 

5 for (each u such that u != v and u has at least one incoming edge) 
6 for (each <i, u> in the graph) 
7 

8 


; if (dist [x ) > dist [i] + length [i \[w)) dist (u | = dist [i] + length [i ]{u); 


Program 6.9: Bellman and Ford algorithm to compute shortest paths 


One) when adjacency lists are used. The observed complexity of the shortest- 
path algorithm can be reduced by noting that if none of the dist values change on 
one iteration of the for loop of lines 4 to 7, then none will change on successive 
iterations. So, this loop may be rewritten to terminate either after n —1 iterations 
or after the first iteration in which no dist values are changed, whichever occurs 
first. Another possibility is to maintain a queue of vertices i whose dist value 
changed on the previous iteration of the for loop. These are the only values for i 
that need to be considered in line 6 during the next iteration. When a queue of 
these values is maintained, we can rewrite the loop of lines 4 to 7 so that on each 
iteration, a vertex i is removed from the queue, and the dist values of all vertices 
adjacent from i are updated as in line 7. Vertices whose dist value decreases as a 
result of this are added to the end of the queue unless they are already on it. The 
loop terminates when the queue becomes empty. 0 


6.4.3 All-Pairs Shortest Paths 


In the all-pairs shortest-path problem, we are to find the shortest paths between 
all pairs of vertices « and v, « #v. This problem can be solved as n independent 
single-soure/all-destinations problems using each of the n vertices of G as a 
source vertex. If we use this approach on graphs with nonnegative edges, the 
total time taken would be O(23) (or O(n7logs + ne) if Fibonacci heaps are used). 
On graphs with negative edges the mun time will be O(n) if adjacency matrices 
are used and O(n7e) if adjacency lists are used. 

Using the dynamic programming approach to the design of algorithms, we 
can obtain a conceptually simpler algorithm that has complexity O(n3) and 
works even when G has edges with negative length. Like the Bellman and Ford 
algorithm, this algorithm requires that G have no cycles with negative length. 
This algorithm is faster for graphs with negative edges, as long as the graphs 
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have at least c *n edges for some suitable constant c. Its observed mun time is 

also less for dense graphs with nonnegative edge lengths. However, for sparse 

graphs with nonnegative edge lengths, using the single-source algorithm and 

Fibonacci heaps results in a faster algorithm for the all-pairs problem. 

The graph G is represented by its length-adjacency matrix as described for 
function ShortestPath (Program 6.8). Define A*[iJLj} to be the length of the 
shortest path from i to j going through no intermediate vertex of index greater 
than k. Then, A”~! [i }[/} will be the length of the shortest i-to-j path in G, since 
G contains no vertex with index greater than n—1. A~'[i][j] is just length [i]f/), 
since the only i-to-j paths allowed can have no intermediate vertices on them. 

The basic idea in the all-pairs algorithm is to successively penerate the 
matrices A~!, A°, A', ---, A"~!. If we have already generated A*~’, then we 
may generate A* by realizing that for any pair of vertices i and j, one of the fol- 
lowing applies: 

(1) The shortest path from i to j going through no vertex with index greater 
than k does not go through the vertex with index &, so its length is 
Atty). 

(2) The shortest path goes through vertex k. In this case, the path consists of a 
subpath from i to k and another one from k to j. These subpaths must be the 
shortest paths from i to k and from k to j going through no vertex with 
index greater than k—1, so their lengths are A*""[i][k} and A*"'(k JE]. 


The preceding rules yield the following formulas for A‘{iJ[/]: 
AJ] = minfA*" IY], API) + ARTY, k20 


and 
ATTY) = length (U1 


_ The function AllLengths (Program 6.10) computes A*~'[i}[j]. The compu- 
tation is done in place using the array a. The reason this computation can be car- 
tied out in place is that A*[i}[k) = A* [iJ[k] and A*[kJLj) = A*="[k]L7], so the 
in-place computation does not alter the outcome. 


Analysis of AllLengths: This algorithm is especially easy to analyze because 
the looping is independent of the data in the matrix a. The total time for function 
AllLengths is O(n’). An exercise examines the extensions needed to obtain the 
(Gj) paths with these lengths. Some speed-up can be obtained by noticing that 
Le ates for loop needs be executed only when a[i}{k] is smaller than 
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1 void MatrixWDigraph::AliLengths(const int n) 
2 {# length (n )[1] is the adjacency matrix of a graph with 1 vertices. 
3 Halil) is the length of the shortest path between i and j. 


4 for {int i= 0; i <n; i++) 

5 for (int j 

6 afi) = length (i]Uj); 4 copy length into a 

7 for (intk=0;k<n3k++) — // fora path with highest vertex index k 
8 for (i = 05 i <n; i++) # for all possible pairs of vertices 

9 for (int j = 0; j <n; j++) 

0 } if (a li)lk] + a(KIUD <a LU) alU) = ati lik) + aks 


Program 6.10: All-pairs shortest paths 


Example 6.7: For the digraph of Figure 6.32(a), the initial a matrix, A~', plus its 
value after each of three iterations, A°, A', and A?, is also given in Figure 6.32. 
ia) 


6.4.4 Transitive Closure 


We end this section by studying a problem that is closely related to the all-pairs 
shortest-path problem. Assume that we have a graph G with unweighted edges. 
‘We want to determine if there is a path from i to j for all values of ¢ and j. Two 
cases are of interest. The first case requires positive path lengths; the second 
requires only nonnegative path lengths. These cases are known as the transitive 
closure and reflexive transitive closure of a graph, respectively. We define them 
as follows: 


Definition: The transitive closure matrix, denoted A*, of a graph, G, is a matrix 
such that A*{iJ[j] = 1 if there is a path of length > 0 from i to J; otherwise, 
A‘(il/)=0.0 


Definition: The reflexive transitive closure matrix, denoted A *, of a graph, G, is 
a matrix such that A°[iJ]Lj] = 1 if there is a path of length 2 0 from / to j; other- 
wise, A“[i]§j]=0.0 


Figure 6.33 shows A* and A* for a digraph. Clearly, the only difference 
between A* and A* is in the terms on the diagonal. A*[i][/] = 1 iff there is a 
cycle of length >1 containing vertex i, whereas A *[i][é] is always one, as there is 
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Figure 6.32; Example for all-pairs shortest-paths problem 


a path of length 0 from i to i. 

We can use function AliLengths (Program 6.10) to compute A*. We begin 
with length [i][j]= 1 if <i,j> is an edge in G and length[iJ[j] = LARGE if 
<é,j> is not in G. When AllLengths terminates, we can obtain A* from the final 
matrix a by letting A*[é}Lj} = | iffa[éJlj] < LARGE. A* can be obtained from 
A * by setting all diagonal elements equal to 1. The total time is O(n). 

Some simplification is achieved by slightly modifying function AliLengths. 
In this modification, we make length and a@ artays of type bool. Initially, 
Jength [i }Lj] = true iff <i,j> is an edge of G. That is, length is the adjacency 
matrix of the graph. Line 10 of AllLengths is replaced by 


aU) = aI] alk] && alkIUD) 


Upon termination of AliLengths, the final matrix a is A*. 

The transitive closure of an undirected graph G can be found more easily 
from its connected components. From the definition of a connected component, 
it follows that there is a path between every pair of vertices in the component 
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Figure 6.33: Graph G and its adjacency matrix A, A*, and A* 


and there is no path in G between two vertices that are in different components. 
Hence, if A is the adjacency matrix of an undirected graph (i.e., A is symmetric) 
then its transitive closure A* may be determined in O(1*) time by first determin- 
ing the connected components of the graph. A*{i]L/) = | iff there is a path from 
vertex i to j. For every pair of distinct vertices in the same component, 
A*[ilfj| = 1. On the diagonal, A*[i}{i] = 1 iff the component containing i has 
at least two vertices. 


EXERCISES 


1. Let Tbe a tree with soot v. The edges of T are undirected. Each edge in T 
has a nonnegative length. Write a C++ function to determine the length of 
the shortest paths from v to the remaining vertices of T, Your function 
should have complexity O(n), where n is the number of vertices in T. Show 
that this is the case. 
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2. Let G be a directed, acyclic graph with n vertices. Assume that the vertices 
are numbered 0 through —I and that all edges are of the form <i,j>, 
where i <j. Assume that the graph is available as a set of adjacency lists 
and that each edge has a length (which may be negative) associated with it. 
Write a C++ function to determine the length of the shortest paths from 
vertex 0 to the remaining vertices. The complexity of your algorithm 
should be O( + e), where e is the number of edges in the graph, Show 
that this is the case. 

3. (a) Do the previous exercise, but this time find the length of the longest 

paths instead of the shortest paths. 
(b) Extend your algorithm of (a) to determine a longest path from vertex 
0 to each of the remaining vertices. 

4, What is a suitable value for LARGE in the context of function ShortestPath 
(Program 6.8)? Provide this as a function of the largest edge length maxL 
and the number of vertices n. 

5. Using the idea of ShortestPath (Program 6.8), write a C++ function to find 
a minimum-cost spanning tree whose worst-case time is O{n7). 

6. Use ShortestPath (Program 6.8) to obtain, in nondecreasing order, the 
lengths of the shortest paths from vertex 0 to all remaining vertices in the 
digraph of Figure 6.34. 


Figure 6.34: A digraph 


7. Rewrite ShortestPath (Program 6.8) under the following assumptions: 
(a) G is represented by its adjacency lists, where each node has three 
fields: vertex, length, and link. length is the length of the correspond- 
ing edge and » the number of vertices in G. 
(b) Instead of S (the set of vertices to which the shortest paths have 
already been found), the set T= V(G)—S is represented using a 
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linked list. 


What can you say about the computing time of your new function relative 
to that of ShortestPath? 


8. Modify ShortestPath (Program 6.8) so that it obtains the shortest paths, in 
addition to the lengths of these paths. What is the computing time of your 
modified function? 


9. Using the directed graph of Figure 6.35, explain why ShortestPath will not 
work properly. What is the shortest path between vertices 0 and 6? 


Figure 6.3: Directed graph on which ShortestPath does not work properly 


10. Prove the correctness of function BellmanFord (Program 6.9). Note that 
this function does not faithfully implement the computation of the 
recurrence for dist. In fact, for k <n~1, the dist values following itera- 
tion k of the for loop of lines 4 to 7 may not be dist*, 


11. Transform function BellmanFord into a complete C++ function. Assume 
that graphs are represented using adjacency lists in which each node has an 
additional field called length that gives the length of the edge represented 
by that node. As a result of this, there is no length-adjacency matrix. Gen- 
erate some test graphs and test the correctness of your function. 

12. Rewrite function BellmanFord so that the loop of lines 4 to 7 terminates 
either after m—1 iterations or after the first iteration in which no dist values 
are changed, whichever occurs first. 

13. Rewrite function BellmanFord by replacing the loop of lines 4 to 7 with 
code that uses a queue of vertices that may potentially result in a reduction 
of other dist vertices. This queue initially contains all vertices that are 
adjacent from the source vertex v. On each successive iteration of the new 
lvop, a vertex i is removed from the queue (unless the queue is empty), and 
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the dist values to vertices adjacent from i are updated as in line 7 of Pro- 

gram 6.9. When the dist value of a vertex is reduced because of this, it is 

added to the queue unless it is already on the queue. 

(a) Prove that the new function produces the same results as the original 
one. 

(b) Show that the complexity of the new function is no more than that of 
the original one. 

Compare the run-time performance of the Bellman and Ford functions of 

the preceding two exercises and that of Program 6.9. For this, generate test 

graphs that will expose the relative performance of the three functions. 

Modify function BellmanFord so that it obtains the shortest paths, in addi- 

tion to the lengths of these paths. What is the computing time of your func- 

tion? 

What is a suitable value for LARGE in the context of function AllLengths 

(Program 6.10)? Provide this as a function of the largest edge length maxL 

and the number of vertices 7. 

Modify function AliLengths (Program 6.10) so that it obtains a shortest path 

for all pairs of vertices. What is the computing time of your new function? 

Use function AilLengths to obtain the lengths of the shortest paths between 

all pairs of vertices in the graph of Figure 6.34. Does AllLengths give the 

right answers? Why? 

By considering the complete graph with n vertices, show that the maximum 

number of simple paths between two vertices is O((n - 1)!). 

Show that A* = A*x A, where matrix multiplication of the two matrices is 

defined as af = vji-,aj4*aajj. Vv is the logical or operation, and a is the 

logical and operation. 

Obtain the matrices A* and A * for the digraph of Figure 6.16. 

What is a suitable value for LARGE when AllPaths (Program 6.8) is used to 

compute the transitive closure of a directed graph? Provide this as a func- 

tion of the number of vertices n. 


ACTIVITY NETWORKS. 


Activity-on- Vertex (AOV) Networks 


All but the simplest of projects can be subdivided into several subprojects called 
activities, _The successful completion of these activities results in the completion 
of the entire project. A student working toward a degree in computer science 
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must complete several courses successfully. The project in this case is to com- 
plete the major, and the activities are the individual courses that have to be 
taken. Figure 6,36 lists the courses needed for a computer science major at a 
hypothetical university. Some of these courses may be taken independently of 
others: other courses have prerequisites and can be taken only if all the prere- 
quisites have already been taken. The data structures course cannot be started 
until certain programming and math courses have been completed. Thus, prere- 
quisites define precedence relations between courses. The relationships defined 
may be more clearly represented using a directed graph in which the vertices 
represent courses and the directed edges represent prerequisites. 


Definition: A directed graph G in which the vertices represent tasks or activities 
and the edges represent precedence relations between tasks is an activity-on- 
vertex network or AOV network, 0 


Figure 6.36(b) is the AOV network corresponding to the courses of Figure 
6.36(a). Each edge <i, > implies that course i is a prerequisite of course j. 


Definition: Vertex i in an AOV network G is a predecessor of vertex j iff there is 
a directed path from vertex i to vertex j. i is an immediate predecessor of j iff 
<i,j> is an edge in G. If i is a predecessor of j, then j is a successor of i. If i is 
an immediate predecessor of j, then j is an immediate successor of i. O 


C3 and C6 are immediate predecessors of C7. C9, C10, C12, and C13 are 
immediate successors of C7. C14 is a successor, but not an immediate succes- 
sor, of C3. 


Definition: A relation - is transitive iff it is the case that for all triples i,j,k, i-f 
and j-k=9i-k. A relation - is irreflexive on a set S if for no element x in S is it the 
case that x-x. A precedence relation that is both transitive and irreflexive is a 
partial order. 0 


Notice that the precedence relation defined by course prerequisites is tran- 
sitive. That is, if course i must be taken before course j (as i is a prerequiste of 
j), and if j must be taken before k, then i must be taken before k. This fact is not 
obvious from the AOV network. For example, <C4, C5> and <CS, C6> are 
edges in the AOV network of Figure 6.36(b). However, <C4, C6> is not. Gen- 
erally, AOV networks are incompletely specified, and the edges needed to make 
the precedence relation transitive are implied. 

If the precedence relation defined by the edges of an AOV network is not 
irreflexive, then there is an activity that is a predecessor of itself and so must be 
completed before it can be started. This is clearly impossible. When there are 
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Course number _ Course name Prerequisites 
cl Programming I None 
C2 Discrete Mathematics None 
c3 Data Structures Ci, C2 
C4 Calculus L None 
cs Calculus ff c4 

C6 Linear Algebra cs 

C7 Analysis of Algorithms C3, C6 
C8 Assembly Language C3 

cy Operating Systems C7, C8 
clo Programming Languages C7 
cil Compiler Design C10 
Cci2 Artificial Intelligence c7 
C13 Computational Theory C7 
cl4 Parallel Algorithms Ci3 
cis Numerical Analysis cS 


(a) Courses needed for a computer science degree at a hypothetical university 


oe 
“© eS © 


{b) AOV network representing courses as vertices and prerequisites as edges 


Figure 6.36: An activity-on-vertex (AOV) network 


no inconsistencies of this type, the project is feasible. Given an AOV network, 
one of our concerns would be to determine whether or not the precedence rela- 
tion defined by its edges is irrefiexive. This is identical to determining whether 
or not the network contains any directed cycles. A directed graph with no 
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directed cycles is an acyclic graph. Our algorithm to test an AOV network for 
feasibility will also generate a linear ordering, vo,v), ***, V»—1. of the vertices 
(activities). This linear ordering will have the property that if vertex i is a prede- 
cessor of j in the network, then i precedes j in the linear ordering. A linear order- 
ing with this property is called a topological order. 


Definition: A topological order is a linear ordering of the vertices of a graph 
such that, for any two vertices i and j, if i is a predecessor of j in the network, 
then i precedes j in the linear ordering. 0 


There are several possible topological orders for the network of Figure 
6.36(b). Two of these are 


C1, C2, C4, C5, C3, C6, C8, C7, C10, C13, C12, C14, C15, C11, C9 
and 
C4, CS, C2, Cl, C6, C3, C8, C15, C7, C9, C10, C11, C12, C13, C14 


If a student were taking just one course per term, then she or he would have to 
take them in topological order. If the AOV network represented the different 
tasks involved in assembling an automobile, then these tasks would be carried 
out in topological order on an assembly line. The algorithm to sort the tasks into 
topological order is straightforward and proceeds by listing a vertex in the net- 
work that has no predecessor. Then, this vertex together with all edges leading 
out from it is deleted from the network. These two steps are repeated until all 
vertices have been listed or all remaining vertices in the network have predeces- 
sors, and so none can be removed. In this case there is a cycle in the network, 
and the project is infeasible. The algorithm is stated more formally in Program 
6.11. 


1 Input the AOV network. Let » be the number of vertices. 
2 for (int i = 0; <7; i++) / output the vertices 

3 

4 if (every vertex has a predecessor) return; 

BE) i network has a cycle and is infeasible. 

6 pick a vertex v that has no predecessors 

7 cout<<v; 

8 delete v and atl edges leading out of v from the network; 
9 


} 


Program 6.11: Design of a topological sorting algorithm 


Example 6.8: Let us try out our topological sorting algorithm on the network of 
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Figure 6.37(a). The first vertex to be picked in line 6 is 0, as it is the only one 
with no predecessors. Vertex 0 and the edges <0, I>, <0, 2>, and <0, 3> are 
deleted. In the resulting network (Figure 6.37(b)), vertices 1, 2, and 3 have no 
predecessor. Any of these can be the next vertex in the topological order. 
Assume that 3 is picked. Deletion of vertex 3 and the edges <3, 5> and <3, 4> 
results in the network of Figure 6.37(c). Either 1 or 2 may be picked next. Fig- 
ure 6.37 shows the progress of the algorithm on the network. O 


® @ 
OD BO 
OG 6) 


(a) Initial (b) Vertex O deleted (c) Vertex 3 deleted 


“eee 


(d) Vertex 2 deleted (e) Vertex 5 deleted (f) Vertex 1 deleted 
Topological order generated: 0, 3, 2,5.1,4 


Figure 6.37: Action of Program 6.1! on an AOV network (shaded vertices 
Tepresent candidates for deletion) 


To obtain a complete algorithm that can be easily translated into a com- 
puter program, it is necessary to specify the data Tepresentation for the AOV net- 
work, _The choice of a data representation, as always, depends on the functions 
you wish to perform. In this problem, the functions are 


(1) decide whether a vertex has any predecessors (line 4) 
(2) delete a vertex together with all its incident edges (line 8) 


To perform the first function efficiently, we maintain a count of the number 
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of immediate predecessors each vertex has. The second function is easily impte- 
mented if the network is represented by its adjacency lists. Then the deletion of 
all edges leading out of vertex v can be carried out by decreasing the predecessor 
count of all vertices on its adjacency list. Whenever the count of a vertex drops 
to zero, that vertex can be placed onto a list of vertices with a zero count. Then 
the selection in line 6 just requires removal of a vertex from this list. 

Asa result of the preceding analysis, we represent the AOV network using 
adjacency lists. We assume that the adjacency representation defines a data 
member count, which is of type int* and that count [i] has been initialized to the 
in-degree of vertex i, OSi<n. This can be done easily at the time of input. 
When edge <i,j> is input, the count of vertex j is incremented by 1. Figure 
6.38(a) shows the internal representation of the network of Figure 6.37(a). 


count _ first data link 


Figure 6.38: Internal representation used by topological sorting algorithm 


Inserting these details into Program 6.11, we obtain the C++ function Topo- 
logicalOrder (Program 6.12). The fist of vertices with zero count is maintained 
as a custom stack. A queue could have been used instead, but a stack is slightly 
simpler. The stack is linked through the count field of the header nodes, since 
this field is of no use after a vertex’s count has become zero. 


Analysis of TopologicalOrder: As a result of a judicious choice of data struc- 
tures, the algorithm is very efficient. For a network with n vertices and e edges, 
the loop of lines 4 and 5 takes O(n) time; lines 6 to 10 take O(r) time over the 
entire algorithm; and the while loop of lines 11 to 15 takes time O(d,) for each 
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1 void LinkedDigraph::TopologicalOrder() / 
2 {/ The n vertices of a network are listed in topological order. 


3 int top ; 7 ‘ \ 
4 for (int i = 03 i <n; i++) H create a linked stack of vertices with 
5 if (count [i] == 0) { count [i] = top; top = i;} / no predecessors 


6 for ((=0; i <n; i++) 
7 if (op == —1) throw " Network has a cycle."; 
8 int j = top; top = count [top }; / unstack a vertex 


9 cout << j << endl; 
10 Chain <int>::Chainlterator ji = adjLists [j}.begin 0% y 
1 while (ji) { // decrease the count of the successor vertices ofj 
12 count [*ji }--5 . 
13 if (count [*ji ] == 0) { count [*ji] = top; top = *jis} # add *ji to stack 
4 jitt; 
15 
16} 


Program 6.12: Topological order 


vertex i, where d; is the out-degree of vertex i. Since this loop is encountered 
once for each vertex output, the total time for this part of the algorithm is 
O((zP4d,)+n) = Oe +n). Hence, the asymptotic computing time of the func- 
tion is O(e +”). It is linear in the size of the problem! D 


6.5.2 Activity-on-Edge (AOE) Networks 


An activity network closely related to the AOV network is the activity-on-edge, 
or AOE, network. The tasks to be performed on a project are represented by 
directed edges. Vertices in the network represent events, Events signal the com- 
pletion of certain activities. Activities represented by edges leaving a vertex 
cannot be started until the event at that vertex has occurred. An event occurs 
only when all activities entering it have been completed. Figure 6.39(a) is an 
AOE network for a hypothetical project with 11 tasks or activities: a), «++, @y)- 
There are nine events: 0, 1, ---, 8. The events 0 and 8 may be interpreted as 
“‘start project’ and ‘‘finish project,” respectively. Figure 6.39(b) gives interpre- 
tations for some of the nine events. The number associated with each activity is 
the time needed to perform that activity. Thus, activity a; requires 6 days, 
whereas ay; requires 4 days. Usually, these times are only estimates. Activities 
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1, a2, and a3 may be carried out concurrently after the start of the project. 
Activities a4, a5, and a¢ cannot be started until events 1, 2, and 3, respectively, 
occur. Activities a7 and ag can be carried out concurrently after the occurrence 
of event 4 (i¢., after a4 and as have been completed). If additional ordering 
Constraints are to be put on the activities, dummy activities whose time is zero 
may be introduced. Thus, if we desire that activities a7 and ag not start until 


both events 4 and 5 have occurred, a dummy activity a 12 Tepresented by an edge 
<5,4> may be introduced. 


interpretation 
0 start of project 
1 completion of activity a, 
4 completion of activities a4 and as 
a completion of activities ag and ay 
8 completion of project 


(b) Interpretation of some of the events in the network of (a) 


Figure 6.39: An AOE network 


Activity networks of the AOE type have proved very useful in the perfor- 
mance evaluation of several types of projects. This evaluation includes deter- 
mining such facts about the project as what is the least amount of time in which 
the project may be completed (assuming there are no cycles in the network), 
which activities should be speeded to reduce project length, and so on. 

Since the activities in an AOE network can be carried out in parallel, the 
minimum time to complete the project is the length of the longest path from the 
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Start vertex to the finish vertex (the length of a path is the sum of the times of 
activities on this path). A path of longest length is a critical path. The path 0, 1, 
4,6, 8 is a critical path in the network of Figure 6.39(a). The length of this criti- 
cal path is 18. A network may have more than one critical path (the path 0, 1, 4, 
7, 8 is also critical). 

The earliest time that an event i can occur is the length of the longest path 
from the start vertex 0 to the vertex i. The earliest time that event v4 can occur is 
7. The earliest time an event can occur determines the earliest start time for all 
activities represented by edges leaving that vertex. Denote this time by e(i) for 
activity a;. For example, ¢(7)=e (8)=7. 

For every activity a;, we may also define the latest rime, i(i), that an 
activity may start without increasing the project duration (i.e., length of the long- 
est path from start to finish). In Figure 6.39(a) we have e(6)=5 and /(6)=8, 
e(8)=7 and [(8)=7. 

All activities for which e(#)=/(i) are called critical activities. The 
difference !(i)~e (/) is a measure of the criticality of an activity. It gives the time 
by which an activity may be delayed or slowed without increasing the total time 
needed to finish the project. If activity ag is slowed down to take 2 extra days, 
this will not affect the project finish time. Clearly, all activities on a critical path 
are strategic, and speeding up noncritical activities will not reduce the project 
duration. 

The purpose of critical-path analysis is to identify critical activities so that 
Tesources may be concentrated on these activities in an attempt to reduce project 
finish time. Speeding a critical activity will not result in a reduced project length 
unless that activity is on all critical paths. In Figure 6.39(a) the activity a, is 
critical, but speeding it up so that it takes only 3 days instead of 4 does not 
reduce the finish time to 17 days. This is so because there is another critical path 
(0, 1, 4, 6, 8) that does not contain this activity. The activities a, and a, are on 
all critical paths. Speeding a, by 2 days reduces the critical path length to 16 
days. Critical-path methods have proved very valuable in evaluating project per- 
formance and identifying bottlenecks. 

Critical-path analysis can also be carried out with AOV networks. The 
length of a path would now be the sum of the activity times of the vertices on 
that path. By analogy, for each activity or vertex we could define the quantities 
e(i) and /(2). Since the activity times are only estimates, it is necessary to re- 
evaluate the project during several stages of its completion as more accurate esti- 
mates of activity times become available. These changes in activity times could 
make previously noncritical activities critical, and vice versa. 

: Before ending our discussion on activity networks, let us design an algo- 
rithm to calculate e (i) and {(i) for all activities in an AOE network. Once these 
quantities are known, then the critical activities may easily be identified. Delet- 
ing all noncritical activities from the AOE network, all critical paths may be 
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found by just generating all paths from the start-to-finish vertex (all such paths 
will include only critical activities and so must be critical, and since no noncriti- 
cal activity can be on a critical path, the network with noncritical activities 
removed contains all critical paths present in the original network). 


6.5.2.1 Calculation of Early Activity Times 


When computing the early and late activity times, it is easiest first to obtain the 
earliest event time, ee [j}, and latest event time, fe [j], for all events, j, in the net- 
work. Thus if activity a; is represented by edge <k,/>, we can compute ¢ (i) and 
1(é) from the following formulas: 


e(i)=ee [k] 
and (6.1) 
1(i)=le {1 }-duration of activity a, 


The times ee [j} and Je[j] are computed in two stages: a forward stage and a 
backward stage. During the forward stage we start with e¢ [0]=0 and compute 
the remaining early start times, using the formula 


ee [j }= max {ee [i] + duration of<i,j>} (6.2) 
iePG) 


where P(j) is the set of all vertices adjacent to vertex j. If this computation is 
carried out in topological order, the early start times of all predecessors of j 
would have been computed prior to the computation of ee [j}. So, if we modify 
TopologicalOrder (Program 6.12) so that it returns the vertices in topological 
order (rather than outputs them in this order), then we may use this topological 
order and Eq. 6.2 to compute the early event times. To use Eq. 6.2, however, we 
must have easy access to the vertex set P(j). Since the adjacency list representa- 
tion does not provide easy access to P(j), we make a more major modification to 
Program 6.12. We begin with the ee array initialized to zero and insert the line 
ee [*ji] = max(ee [*ji], ee |] + duration of <j, */i>3 
between lines 12 and 13. This modification results in the evaluation of Eq. (6.2) 
in parallel with the generation of a topological order. ee (j) is updated each time 
the ee (i) of one of its predecessors is known (i.e., when i is ready for output). 

To illustrate the working of the modified TopologicalOrder algorithm, fet 
us try it out on the network of Figure 6.39(a). The adjacency lists for the net- 
work are shown in Figure 6.40(a). The order of nodes on these lists determines 
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output 0 ; 
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(b) Computation of ee 


Figure 6.40: Computing ee using modified TopologicalOrder (Program 6.12) 
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the order in which vertices will be considered by the algorithm. At the outset, 
the early start time for all vertices is 0, and the start vertex is the only one in the 
stack. When the adjacency list for this vertex is processed, the early start time of 
all vertices adjacent from 0 is updated. Since vertices 1, 2, and 3 are now in the 
stack, all their predecessors have been processed, and Eq. (6.2) has been 
evaluated for these three vertices. ee [5] is the next one determined. When ver- 
tex 5 is being processed, ee [7] is updated to 11. This, however, is not the true 
value for ee [7], since Eq. (6.2) has not been evaluated over all predecessors of 7 
(vq has not yet been considered). This does not matter, as 7 cannot get stacked 
until all its predecessors have been processed. ee [4] is next updated to 5 and 
finally to 7. At this point ce [4] has been determined, as all the predecessors of 4 
have been examined. The values of ec [6] and ee [7] are next obtained. ee [8] is 
ultimately determined to be 18, the length of a critical path. You may readily 
verify that when a vertex is put into the stack, its early time has been correctly 
computed. The insertion of the new statement does not change the asymptotic 
computing time; it remains O(e +7). 


6.5.2.2 Calculation of Late Activity Times 


In the backward stage the values of /e[i] are computed using a function analo- 
gous to that used in the forward stage. We start with fe [n-1}=ee (n—1] and use 
the equation : 


le j|= min {le (7) - duration of <j,i>} (63) 
ieSG) 


where S(j) is the set of vertices adjacent from vertex j. The initial values for 
le [i] may be set to ee [n—1]. Basically, Eq. (6.3) says that if <j,i> is an activity 
and the latest start time for event i is /e(/], then event j must occur no Jater than 
le [i] — duration of <j,i>. Before fe [j] can be computed for some event j, the 
latest event time for all successor events (i.e., events adjacent from /) must be 
computed. Once we have obtained the topological order and ee [n—1) from the 
modified version of Program 6.12, we may compute the late event times in 
reverse toplogical order using the adjacency list of vertex j to access the vertices 
in S(j). This computation is shown belaw for our example of Figure 6.39(a). 


fe [8] = ee [8] = 18 

te {6| = min{ie [8] - 2} = 16 

fe [7] = min{le(8}~ 4} = 14 

te 4] = min{le|6)—9, le[7]-7} =7 
te[1] = min{le(4] - 1} =6 

te|2| = minflel4t- 1} =6 
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te (5) = min{le [7] — 4} = 10 
te (3] = min{fe(5] ~ 2} = 8 
te [0] = min{te [1] — 6, te [2] —4, fe [3]~5} =0 


If the forward stage has already been carried out and a topological ordering 
of the vertices obtained, then the values of /e [i] can be computed directly, using 
Eq. (6.3), by performing the computations in the reverse topological order. The 
topological order generated in Figure 6.40(b) is 0, 3, 5, 2, 1, 4, 7, 6, 8. We may 
compute the values of le [i] in the order 8, 6,7, 4, 1, 2,5, 3, 0, as all successors of 
an event precede that event in this order. In practice, one would usually compute 
both ee and le. The procedure would then be to compute ee first, using algorithm 
TopologicalOrder, modified as discussed for the forward stage, and then to com- 
pute /e directly from Eq. (6.3) in reverse topologicat order. 

Using the values of ee (Figure 6.40) and of le (above), and Eq. (6.1), we 
may compute the early and late times e (i) and /(i) and the degree of criticality 
(also called slack) of each task. Figure 6.41 gives the values. The critical activi- 
ties are ay, 24, 47, @g, 21g, and @j,. Deleting all noncritical activities from the 
network, we get the directed graph or critical network of Figure 6.42. All paths 
from 0 to 8 in this graph are critical paths, and there are no critical paths in the 
original network that are not paths in this graph. 


0 (i) 
0 2 No 
0 3 No 
6 0 Yes 
4 2 No 
5 3 No 
7 0 Yes 
ii 0 Yes 
7 3 No 
a19 16 0 Yes 
an 14 0 Yes 


Figure 6.41: Early, late, and criticality values 


As a final remark on activity networks, we note that the function Topologi- 
calOrder detects only directed cycles in the network. There may be other flaws, 
such as vertices not reachable from the start vertex (Figure 6.43). When a 
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Figure 6.43: AOE network with some nonreachable activities 


critical-path analysis is carried out on such networks, there will be several ver- 
tices with ee [i] =0. Since all activity times are assumed > 0, only the start ver- 
tex can have ee [i] = 0. Hence, critical-path analysis can also be used to detect 
this kind of fault in project planning. 


EXERCISES 


1. Does the following set of precedence relations (<) define a partial order on 
the elements 0 thru 4? Why? 


0<1,1<3;1<2;2<3,2<4,4<0 


(b) 
(c) 
(d) 
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For the AOE network of Figure 6.44 obtain the early, e(). and late, 
1(), start times for each activity. Use the forward-backward 
approach. 

Whar is the earliest time the project can finish? 

Which activities are critical? 

Is there any single activity whose speed-up would result in a reduc- 
tion of the project length? 


Figure 6.44; An AOE network 


3. Define a critical AOE network to be an AOE network in which all activi- 
ties are critical. Let G be the undirected graph obtained by removing the 
directions and weights from the edges of the network. 


(a) 


(b) 


Show that the project length can be decreased by speeding up 
exactly one activity if there is an edge in G that lies on every path 
from the start vertex to the finish vertex. Such an edge is called a 
bridge. Deletion of a bridge from a connected graph separates the 
graph into two connected components. 


Write an O(n +e) function using adjacency lists to determine 
whether the connected graph G has a bridge. If G has a bridge, your 
function should output one such bridge. 


4, Write a C++ program that inputs an AOE network and outputs the follow- 


ing: 
(a) 
(b) 


A table of all events together with their earliest and latest times. 


A table of ail activities together with their early and late times. This 
table should also list the slack for each activity and identify all criti- 
cal activities (see Figure 6.41). 
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(c) The critical network. 


(d) Whether or not the project length can be reduced by speeding a sin- 
gle activity. Ifso, then by how much? 


5. Define an iterator class Topolterator for iterating through the vertices of a 
directed acyclic graph in topological order. 


6.6 REFERENCES AND SELECTED READINGS 


Euler's original paper on the Konigsberg bridge problem makes interesting read- 
ing. This paper has been reprinted in: ‘‘Leonhard Euler and the Konigsberg 
bridges,”’ Scientific American, 189:1, 1953, pp. 66-70. 

The biconnected-component algorithm is due to Robert Tarjan. This, 
together with a linear-time algorithm to find the strongly connected components 
of a directed graph, appears in the paper ‘‘Depth-first search and linear graph 
algorithms,” by R. Tarjan, SIAM Journal on Computing, 1:2, 1972, pp. 146-159. 

Prim’s minimum-cost spanning tree algorithm was actually first proposed 
by Jamik in 1930 and rediscovered by Prim in 1957. Since virtually all refer- 
ences to this algorithm give credit to Prim, we continue to refer to it as Prim’s 
algorithm, Similarly, the algorithm we refer to as Sollin’s algorithm was first 
proposed by Boruvka in 1926 and rediscovered by Sollin several years later. For 
an interesting discussion of the history of the minimum spanning tree problem, 
see ‘On the history of the minimum spanning tree problem,”” by R. Graham and 
P. Hell, Annals of the History of Computing, 7:1, 1985, pp. 43-57. 

Further algorithms on graphs may be found in Graphs: Theory and applica- 
tions, by K. Thulasiraman and M. Swamy, Wiley Interscience, 1992. 


6.7 ADDITIONAL EXERCISES 
1. Program 6.13 was obtained by Stephen Barnard to find an Eulerian walk in 

a connected, undirected graph that has no vertices with odd degree. 

(a) Show that if G is represented by its adjacency multilists additions to 
path take O(1) time, then function Euler works in time O(a + @). 

(b) Prove by induction on the number of edges in G that this algorithm 
does obtain an Eulerian walk for all graphs G having such @ walk. 
The initial call to Euler can be made with any vertex v. 


(c) At termination, what has to be done to determine whether or not G 
has an Eulerian walk? 
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virtuat Path Graph::Euler (int v) 


Path path = ( }; 

for ((all vertices w adjacent to v) && (edge (v,w) not yet used)) { 
mark edge (v,w) as used; 
path = {(v,w)} U euler (w) U path; 


return path; 


Program 6.13: Finding an Eulerian walk 


2. A bipartite graph G = (V, E) is an undirected graph whose vertices can be 


as 


Partitioned into two disjoint sets, A and B = V —A, with the following pro- 
perties: (1) No two vertices in A are adjacent in G, and (2) no two vertices 
in B are adjacent in G. The graph G4 of Figure 6.5 is bipartite. A possible 
Partitioning of V is A = {0,3,4,6} and B= (1,2,5,7}. Write an algorithm 
to determine whether a graph G is bipartite. If G is bipartite your algorithm 
should obtain a partitioning of the vertices into two disjoint sets, A and B, 
satisfying properties (1) and (2) above. Show that if G is represented by its 
adjacency lists, then this algorithm can be made to work in time O(n +e), 
where n =|Vjand e =|E|. 
Show that every tree is a bipartite graph. 
Prove that a graph G is bipartite iff it contains no cycles of odd length. 
The radius of a tree is the maximum distance from the root to a leaf. Given 
a connected, undirected graph, write a function to find a spanning tree of 
minimum radius. (Hint: Use breadth-first search.) Prove that your algo- 
rithm is correct. 
‘The diameter of a tree is the maximum distance between any two vertices. 
Given a connected, undirected graph, write an algorithm for finding a span- 
ning tree of minimum diameter. Prove the correctness of your algorithm. 
Let G[n Jl} be a wiring grid. G[i}L/] > 0 represents a grid position that is 
blocked; G{i]Ll/] = 0 represents an unblocked position. Assume that posi- 
tions {a |[b ] and fc }{d] are blocked positions. A path from {a J[>] to [6 }[c] 
is a sequence of grid positions such that 
(a) [a@]{b] and [c |[d] are, respectively, the first and last positions on the 
path 


(b) successive positions of the sequence are vertically or horizontally 
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adjacent in the grid 
(c) all positions of the sequence other than the first and last are 
unblocked positions 
The length of a path is the number of grid positions on the path. We wish 
to connect positions [@][b] and [c ][d} by a wire of shortest length. The 


wire path is a shortest grid path between these two vertices. Lee’s algo- 
rithm for this works in the following steps: 


{a) [Forward step] Start a breadth-first search from position [a][6], 
labeling unblocked positions by their shortest distance from [a]{). 
To avoid conflicts with existing labels, use negative labels. The 
labeling stops when the position [¢ ][d] is reached. 

(b) [Backtrace] Use the labels of (a) to label the shortest path between 
[a}[b] and {c }{d], using the unique label w > 0 for the wire. For this, 
start at position [c ]{d }. 

(c) [Clean-up] Change the remaining negative labels to 0. 


Write algorithms for each of the three steps of Lee's algorithm, What is the 
complexity of each step? 


8. Another way to represent a graph is by its incidence matrix, INC. There is 
one row for each vertex and one column for each edge. Then 
INC[i]L/] = 1 if edge j is incident to vertex i. The incidence matrix for the 
graph of Figure 6.17(a) is given in Figure 6.45. 


ee 


0123456789 


071100000000 
1}1011000000 
2]}0100110000 
3}0010001000 
4;0001000100 
5}0000100010 
6,0000010001 
7TL{O0O000001IIE 


Figure 6.45; Incidence matrix of graph of Figure 6.17(a) 


The edges of Figure 6.17(a) have been numbered from left to right and top 
to bottom. Rewrite function DFS (Program 6.1} so that it works on a graph 
represented by its incidence matrix. 


10. 


11. 
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If ADJ is the adjacency matrix of a graph G =(V,E), and INC is the 
incidence matrix, under what conditions will ADJ =INCx INC? ~I, 
where INC? is the transpose of matrix INC? 1 is the identity matrix, and 
the matrix product C = A x B, where all matrices are 1 xn, is defined as 
Cy = VE handy. v is the Il operation, and a is the && operation. 

An edge (u, v) of a connected, undirected graph G is a bridge iff its dele- 
tion from G results in a graph that is not connected. In the graph of Figure 
6.20, the edges (0, 1), (3, 5), (7, 8), and (7, 9) are bridges. Write an algo- 
rithm that runs in O(n +e) time to find the bridges of G. 7 and ¢ are, respec- 
tively, the number of vertices and edges of G. (Hint: Use the ideas in func- 
tion Biconnected (Program 6.5).) 

(Programming Project] Write a set of C++ classes for manipulating 
graphs. Such a collection should allow input and output of arbitrary 
graphs, determining connected components, spanning trees, minimum-cost 
spanning trees, biconnected components, shortest paths, and so on. You 
should include at least the classes of Figure 6.13. 


CHAPTER 7 


Sorting 


7.4. MOTIVATION 


In this chapter, we use the term list to mean a collection of records, each record 
having one or more fields. The fields used to distinguish among the records are 
known as keys. Since the same list may be used for several different applications, 
the key fields for record identification depend on the particular application. For 
instance, we may regard a telephone directory as a list, each record having three 
fields; name, address, and phone number. The key is usually the person’s name. 
However, we may wish to locate the record corresponding to a given number, in 
which case the phone number field would be the key. In yet another application 
we may desire the phone number at a particular address, so the address field 
could also be the key. s 
One way to search for a record with the specified key is to examine the list 
of records in left-to-right or right-to-left order. Such a search is known as a 
sequential search. Program 7.1 gives a sequential search function that examines 
the records in left-to-right order. We assume that the relational operators (<, > 
==, etc.) have been overloaded so that a comparison between a record of type E 
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and a key & of type K is done by comparing the record key with k. 


template <class £, class K> 
int SeqSearch (E *a, const int n, const K& k) 
{4 Search a[I:n] from left to right. Return feast / such that 
4 the key of a [i] equals k. If there is no such i, retum 0. 
int i; 
for(i = l,i<a=n &&ali}!=k itt); 
if (¢ > n) return 0; 
return i; 


Program 7,1: Sequential search 


If no record in a[1:n] has key value k, the search is unsuccessful. Program 
7.1 makes n key comparisons when the search is unsuccessful. The number of 
key comparisons made in the case of a successful search depends on the position 
of the search key in the array a. If all keys are distinct and the key of a[i} is 
being searched for, then i key comparisons are made. The average number of 
comparisons for a successful search is, therefore, 


Cd )/m=+)/2. 
Isisn 

It is possible to do much better than this when looking up phone numbers. 
The fact that the entries in the list (i.¢., the telephone directory) are in lexico- 
graphic order (on the name key) enables one to look up a number while examin- 
ing only a very few entries in the list. Binary search (see Chapter 1) is one of the 
better-known methods for searching an ordered, sequential list. A binary search 
takes only O(logn) time to search a list with n records. This is considerably 
better than the O(n) time required by a sequential search. We note that when a 
sequential search is performed on an ordered list, the conditional of the for loop 
of SegSearch can be changed to i <=n && ali] <k. This change must be 
accompanied by a change of the conditional i >n toi >n It afi] !=k. These 
changes improve the performance of Program 7.1 for unsuccessful searches. 

Getting back to our example of the telephone directory, we notice that nei- 
ther a sequential nor a binary search strategy corresponds to the search method 
actually employed by humans. If we are looking for a name that begins with the 
letter W, we start the search toward the end of the directory rather than at the 
middie. A search method based on this interpolation scheme would begin by 
comparing k with afi}, where i = ((k —afl]-key)/(a[n].key —a{I].key)) * 2, 
and a[}).key and a{n].key are the smallest and largest keys in the list. An 
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interpolation search can be used only when the list is ordered. The behavior of 
such a search depends on the distribution of the keys in the list. 

We have seen that something is to be gained by maintaining the list in an 
ordered manner if the list is to be searched repeatedly. Let us now look at 
another example in which the use of ordered lists greatly reduces the computa- 
tional effort. The problem we are now concerned with is that of comparing two 
lists of records containing data that are essentially the same but have been 
obtained from two different sources. Such a problem could arise, for instance, in 
the case of the United States Internal Revenue Service (IRS), which might 
receive millions of forms from various employers stating how much they paid 
their employees and then another set of forms from individual employees stating 
how much they received. So we have two lists of records, and we wish to verify 
that there is no discrepancy between the two. Since the forms arrive at the IRS 
in a random order, we may assume a random arrangement of the records in the 
lists. The keys here are the social security numbers of the employees. 

Let /1 be the employer list and /2 the employee list. Let /I{i].key and 
I2[i]. key, respectively, denote the key of the ith record in /1 and /2. We make 
the following assumptions about the required verification: 


(1) If there is no record in the employee list corresponding to a key in the 
employer list, a message is to be sent to the employee. 


(2) If the reverse is true, then a message is to be sent to the employer. 


(3) If there is a discrepancy between two records with the same key, @ message 
to this effect is to be output. 


Function Verify] (Program 7.2) solves the verification problem by directly 
comparing the two unsorted lists. The data type of the records in the list is Ele- 
ment and we assume that the relational operators have been overloaded so that a 
comparison between records is made by comparing their keys and that the output 
operator (<<) has been overloaded to output the record’s key. The function com- 
pare returns true iff the two input records are identical in all fields. The com- 
plexity of Verify! is O(n), where n amd m are, respectively, the number of 
records in the employer and employee lists. On the other hand, if we first sort the 
two lists and then do the comparison, we can carry out the verification task in 
time O(tsurr(!t) + syn) +. + m), where fs,,,(it) is the time needed to sorta list 
of n records. As we shall see, it is possible to sort n records in O(nlogn) time, so 
the computing time becomes O(nax{nlogn, mlogm}). Function Verify2 (Pro- 
gram 7.3) achieves this time. ; : 

We have seen two important uses of sorting: (1) as an aid in searching and 
(2) as a means for matching entries in lists. Sorting also finds application in the 
solution of many other more complex problems from areas such as optimization, 
graph theory and job scheduling. Consequently, the problem of sorting has great 
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void Verify (Element *11, Element *!2, const int n, const int m) ; 

{4 Compare two unordered lists /1 and 12 of size » and m, respectively. 
bool marked = new bool [m + 1]; 
fill(marked +1, marked +m + 1, false); 


for (i = 1; i <= n; i++) 


int j = SeqSearch (12, m, N[E); : 
if (j == 0) cout << /![i} <<” not in /2" << endl; // satisfies (1) 
else 


if (!Compare (L1[i], (2[j }) # satisfies (3) 
cout << “Discrepancy in" << /1[i] << endl; 
marked [j ] = true; // mark /2[j] as being seen 


} 
for (i = 1; i <= m; i++) . 
if (! marked [i ]) cout << [2[i} <<" not in /1. "<< endl; / satisfies (2) 
; delete [ ] marked ; 


Program 7.2: Verifying two lists using a sequential search 


felevance in the study of computing. Unfortunately, no one sorting method is the 
best for all applications. We shall therefore study several methods, indicating 
when one is superior to the others. 

First let us formally state the problem we are about to consider. We are 
given a list of records (R,, Ro, -*-, R,)- Each record, R;, has key value K;,. In 
addition, we assume an ordering relation (<) on the keys so that for any two key 
values x and y, x = y or x <y or y <x. The ordering relation (<) is assumed to 
be transitive (i.e., for any three values x, y, and z, x < y and y <z implies x < z). 
The sorting problem then is that of finding a permutation, 0, such that 
Koi) S Kast). 1S i Sn —1. The desired ordering is (Roy, Roy °°» Roway)- 

Note that when the list has several key vatues that are identical, the permu- 
tation, 6, is not unique. We shall distinguish one permutation, 6,, from the oth- 
ers that also order the list. Let 6, be the permutation with the following proper- 
ties: 


(1) Koy S$ Kosi) USisa-1. 
(2) Ifi<jand K; 


K; in the input list, then R, precedes R; in the sorted fist. 
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void Verify2(Element ¥11, Element 12, const int n, const int m) 
{/ Same task as Verify. However, this time we first sort 71 and /2. 
Sort(l, 2); # sort into increasing order of key 
Sort(t2, m); 
inti=1,j=1; 
while (i <= n) && (j <= m)) 
fC [i] </) 
{ 


cout << /I[i] << " not in /2" << endl; 
i++; 


} 
ra if fi] >/U)) 


cout << 12[j] <<" not in 11" << endl; 
Sts 
) 
else 
{// equal keys 
if (!}Compare (11[i}, (2(])) 
cout << "Discrepancy in " << /I[i] <<endl; 
i+t; j++ 


} 
if (i <= n) OurputRest(\, i, n, 1); # output records i through n of 11 
else if (j <= m) OutputRest(I2, j, m, 2); / output records j through m of 12 


Program 7.3: Fast verification of two lists 


A sorting method that generates the permutation 6, is stable. 

We characterize sorting methods into two broad categories: (1) internal 
methods {i.e., methods to be used when the list to be sorted is small enough so 
that the entire sort can be carried out in main memory) and (2) external methods 
(ie., methods to be used on larger lists). The following internal sorting methods 
will be developed: Insertion Sort, Quick Sort, Merge Sort, Heap Sort, and Radix 
Sort. This development will be followed by a discussion of external sorting. 
Throughout, we assume that relational operators have been overloaded so that 
record comparison is done by comparing their keys. 
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7.2 INSERTION SORT 


The basic step in this method is to insert a new record into a sorted sequence of 
tecords in such a way that the resulting sequence of size i + I is also ordered. 
Function /nsert (Program 7.4) accomplishes this insertion. 


template <class 7> 
void Insert(const T& e, T *a, int i) 
{// Insert e into the ordered sequence a[1:i] such that the 
H resulting sequence a[1:i +1] is also ordered. 
/ The array a must have space allocated for at least i + 2 elements. 
a(0) =e; 
while (e < a[i]) 


alit+l]=ali}s 


> 


ali+l)=e; 


Program 7.4: Insertion into a sorted list 


The use of a[0] enables us to simplify the while loop, avoiding a test for 
end of list (i.e., i< 1). In Insertion Sort, we begin with the ordered sequence 
afl] and successively insert the records a(2], a{3], ---,a{n]. Since each inser- 
tion leaves the resultant sequence ordered, the list with n records can be ordered 
making n — 1 insertions. The details are given in function InsertionSort (Pro- 
gram 7.5). 


Analysis of InsertionSort: In the worst case Insert(e, a, i) makes i + 1 com- 
Parisons before making the insertion. Hence the complexity of Insert is O(i). 
InsertionSort invokes Insert for i = j-1 = 1,2, -+-,n—-1. So, the complexity 
of InsertionSort is 7 

ant 
OCD +1) = O(n?). 

f=) 

We can also obtain an estimate of the computing time of Insertion Sort 
based on the relative disorder in the input list. Record R; is left our of order 
(LOO) iff R; < max (Rj }. The insertion step has to be cartied out only for those 

<i 


ba 
records that are LOO. If & is the number of LOO records, the computing time is 
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template <class T> 
void InsertionSort(T *a, const int 2) 
{/f Sort a[1:2) into nondecreasing order. 
for (int j = 2; j <= nj j++) { 
Tremp =alj); 
Insert(temp, a, j-1)3 


} 


Program 7.5: Insertion Sort 


O((k + 1)n) = Olkn). We can show that the average time for InsertionSort is 
O(n*) as well. O 


Example 7.1: Assume that x = 5 and the input key sequence is 5, 4, 3, 2, 1. 
After each insertion we have 


For convenience, only the key field of each record is displayed, and the 
sorted part of the list is shown in bold. Since the input list is in reverse order, as 
each new record is inserted into the sorted part of the list, the entire sorted part is 
shifted right by one position. Thus, this input sequence exhibits the worst-case 
behavior of Insertion Sort. 


Example 7.2: Assume that n = 5 and the input key sequence is 2, 3, 4, 5,1. 
After each iteration we have 
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In this example, only record 5 is LOO, and the time for each j = 2, 3, and 4 is 
O(1), whereas for j = 5 it is O(a). 0 


It should be fairly obvious that /nsertionSort is stable. The fact that the 
computing time is O(kn) makes this method very desirable in sorting sequences 
in which only a very few records are LOO {(i.e., k<<n). The simplicity of this 
scheme makes it about the fastest sorting method for small n (say, n S 30). 


Variations 


1. Binary Insertion Sort: We can reduce the number of comparisons made in an 
insertion sort by replacing the sequential searching technique used in Insert (Pro- 
gcam 7.4) with binary search. The number of record moves remains unchanged. 


2. Linked Insertion Sort: The elements of the list are represented as a linked list 
rather than as an array. The number of record moves becomes zero because only 
the link fields require adjustment. However, we must retain the sequential search 
used in Insert. 


EXERCISES 


1. Write the status of the list (12, 2, 16, 30, 8, 28, 4, 10, 20, 6, 18) at the end of 
each iteration of the for loop of /nsertionSort (Program 7.5). 

2. Write a C++ function that implements Binary Insestion Sort. What is the 
worst-case number of comparisons made by your sort function? What is 
the worst-case number of record moves made? How do these compare with 
the corresponding numbers for Program 7.5? 

3. Write a C++ function that implements Linked Insertion Sort. What is the 
worst-case number of comparisons made by your sort function? What is 
the worst-case number of record moves made? How do these compare with 
the corresponding numbers for Program 7.5? 
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7.3 QUICK SORT 


We now turn our attention to a sorting scheme with very good average behavior. 
The Quick Sort scheme developed by C. A. R. Hoare has the best average 
behavior among the sorting methods we shall be studying. In Quick Sort, we 
select a pivot record from among the records to be sorted. Next, the records to 
be sorted are reordered so that the keys of records to the left of the pivot are less 
than or equal to that of the pivot and those of the records to the right of the pivot 
are greater than or equal to that of the pivot. Finally, the records to the left of the 
pivot and those to its right are sorted independently (using the Quick Sort 
method recursively). 

Program 7.6 gives the resulting Quick Sort function. To sort a[l:n], the 
function invocation is QuickSort(a, 1,7). Function QuickSort assumes that 
a[n + 1] has been set to have a key at least as large as the remaining keys. 


template <class T> 
void QuickSort(T *a, const int left, const int right) 
{#/ Sort a [left: right] into nondecreasing order. 
H a [left] is arbitrarily chosen as the pivot. Variables i and j 
/ are used to partition the subarray so that at any time a {mm} < pivot, m <i, 
Hand a(m)2 pivot, m > j. It is assumed that a [left] < a [right + 1). 
if (left < right) { 
int i = left, 
jaright +1, 
pivot =a [left); 
do{ 
do i+ +; while (a [i] < pivor); 
do j—-; while (a [j] > pivot); 
if (§< j) swap(ali),aU Ds 
} while (i <j); 
swap(a (left), aL 1)s 


QuickSort(a, left, j-1)3 
QuickSort(a, j +1, right); 


} 


Program 7.6: Quick Sort 


Example 7.3: Suppose we are to sort a list of 10 records with keys (26, 5, 37, 1. 
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61, 11, 59, 15, 48, 19). Figure 7.1 gives the status of the list at each call of 
QuickSort. Square brackets indicate sublists yet to be sorted. 0 


Ry Rg Rs Re Ry Rg Ro Ryo left 

37 tks (‘és ST 

19 1 15) 26 {59 61 48 37) 1 

11 19 15) 26 [59 61 48 37 1 

11 [19 15) 26 [59 61 48 37) 5 
7 
10 


S 


iW 15 19 26 (S9 61 48 37} 
11 15 19 26 [48 37) 59 [61) 
11 1S 19 26 37 48 59 [63) 
11 15 19 26 37 48 59 61 


AUANUBuu® 


Figure 7.1: Quick Sort example 


Analysis of QuickSort: The worst-case behavior of QuickSort is examined in 
Exercise 2 and shown to be O(n). However, if we are lucky, then each time a 
record is correctly positioned, the sublist to its left will be of the same size as 
that to its right. This would leave us with the sorting of two sublists, each of size 
roughly 1/2. The time required to position a record in a list of size nis O(n). If 
T(n) is the time taken to sort a list of n records, then when the list splits roughly 
into two equal parts each time a record is positioned correctly, we have 


T(n) Scn + 2T(n2), for some constant ¢ 
Sen + Acn/2 + 2T(n/)) 
S2cn + 4T(n/) 


Sen logon + nT(1) = O(n logn) 


Lemma 7.1 shows that the average computing time for function QuickSort 
is O(n logit). Moreover, experimental results show that as far as average com- 
puting time is concerned, Quick Sort is the best of the internal sorting methods 
we shall be studying. 


Lemma 7.1: Let T,,,.(n) be the expected time for function QuickSort to sort a 
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list with n records. Then there exists a constant & such that Tayg(n) S knlog.n for 
n22. 


Proof: In the call to QuickSort (list ,1,n), the pivot gets placed at position j. 
This leaves us with the problem of sorting two sublists of size j — 1 and n —j. 
The expected time for this is Taye(j ~1)+ Tavg(n —j). The remainder of the 
function clearly takes at most cn time for some constant c. Since j may take on 
any of the values 1 to 1 with equal probability, we have 


0 n-l 
Tavg(t) Sn + iL ¥ Tavg(i 1) + Taug(t —j)) = en + 2 ETovg) (7-1) 
N jay n j=0 


forn 22. We may assume Tyyp(0) < b and Toyg(1) $b for some constant b. We 
shall now show Tyyg(n) S kniogen for n 22 and k = 2(b +c). The proof is by 
induction on nr. 


Induction base: For n = 2, Bq. (7.1) yields Tag(2) $ 2c + 2b < knlog,2. 
Induction hypothesis: Assume Tyye(n) Sknlogen for 1 Sn < m. 
Induction step: From Eq. (7.1) and the induction hypothesis we have 


4b aml 7 4b | 2km) F 
ee, =&- ar 72) 
Tayg(n) S cm + ~~ + Zinsom + + Bi logei (7.2) 


Since jlog,j is an increasing function of j, Eq. (7.2) yields 


n 
4b | 2k 4b 2k 
Taygm) Sem + im + oy totes dx=cm+ ne + 
=om + & + kmlog,m — - <kmlog.m, form220 
m 


Unlike Insertion Sort, where the only additional space needed was for one 
record, Quick Sort needs stack space to implement the recursion. If the lists split 
evenly, as in the above analysis, the maximum recursion depth would be log n 
requiring a stack space of O(log n). The worst case occurs when the list is split 
into a left sublist of size n - 1 and a right sublist of size 0 at each level of recur- 
sion. In this case, the depth of recursion becomes n, requiring stack space of 
O(n). The worst-case stack space can be reduced by a factor of 4 by realizing 
that right sublists of size less than 2 need not be stacked. An asymptotic reduc- 
tion in stack space can be achieved by sorting smaller sublists first. In this case 
the additional stack space is at most O(log #1). 
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Variation—Quick Sort using a median-of-three: Our version of Quick Son 
always picked the key of the first record in the current sublist as the pivot. A 
better choice for this pivot is the median of the first, middle, and last keys in the 
current sublist. Thus, pivot = median {K;, Kassy2 K,}. For example, 
median{10, 5,7} =7 and median{10,7, 7} =7. 


EXERCISES 


1, Drawa figure similar to Figure 7.1 starting with the list (12, 2, 16, 30, 8, 28, 
4, 10, 20, 6, 18). 
2. (a) Show that QuickSort takes O(n 2) time when the input list is already 
in sorted order. 
(b) Show that the worst-case time complexity of QuickSort is O(n?). 
(c) Why is list (left) < list [right + 1 required in QuickSort? 
3. (a) Write a nonrecursive version of QuickSort incorporating the 
median-of-three rule to determine the pivot key. 
(b) Show that this function takes O(n log 2) time on an already sorted 
list. 
4. Show that if smaller sublists are sorted first, then the recursion in QuickSort 
can be simulated by a stack of depth O(log 7). 
5. Quick Sort is an unstable sorting method. Give an example of an input list 
in which the order of records with equal keys is not preserved. 
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Both of the sorting methods we have seen so far have a worst-case behavior of 
O(n?). It is natural at this point to ask the question, What is the best computing 
time for sorting that we can hope for? The theorem we shall prove shows that if 
we restrict our question to sorting algorithms in which the only operations per- 
mitted on keys are comparisons and interchanges, then O(n log m) is the best 
possible time. 

The method we use is to consider a tree that describes the sorting process. 
Each vertex of the tree represents a key comparison, and the branches indicate 
the result. Such a tree is called a decision tree. A path through a decision tree 
represents a sequence of computations that an algorithm could produce. 


Example 7.4: Let us look at the decision tree obtained for Insertion Sort work- 
ing on a list with three records (Figure 7.2), The input sequence is Ry, Ro, and 
R3, so the root of the tree is labeled [1, 2, 3}. Depending on the outcome of the 
comparison between keys K; and K >», this sequence may or may not change. If 
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K2<XK,, then the sequence becomes [2, 1, 3]; otherwise it stays (1, 2, 3]. The 
full tree resulting from these comparisons is given in Figure 7.2. 


Ki sKQ2,3) 
Yes No 
(231K $k} &s SK) 21,3) 
Yes No Yes No 
sah Doe po ae NS. 
(1.2.3K stop) &1 SK U3.2I213K stop) Kz $ 32.3.1] 
“7 aia WT ag 
Yes No Yes No 


7 A. : ae 
113.2 stop » & stop Y13,1,2112,3,1K stop) a stop 13.2.1] 
a “nt “vo “VE 
Figure 7.2: Decision tree for Insertion Sort 


The leaf nodes are labeled I to VI. These are the only points at which the 
algorithm may terminate. Hence, only six permutations of the input sequence 
are obtainable from this algorithm. Since all six of these are different, and 3! = 6, 
it follows that this algorithm has enough leaves to constitute a valid sorting algo- 
rithm for three records. The maximum depth of this tree is 3. Figure 7.3 gives 
six different orderings of the key values 7, 9, and 10, which show that all six per- 
mutations are possible. O 


Theorem 7.1: Any decision tree that sorts n distinct elements has a height of at 
least logo(!) + 1. 


Proof: When sorting n elements, there are n! different possible results. Thus, 
every decision tree for sorting must have at least 7! leaves. But a decision tree 1s 
also a binary tree, which can have at most 2‘! leaves if its height is k. There- 
fore, the height must be at least logy 2! +1. 0 


Corollary: Any algorithm that sorts only by comparisons must have a worst- 
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ofl sample input key values that 

leaf | permutation ive the permutation 

T 123 7.9, 10] 

IL 132 (7, 10, 9] 
Tl 312 (9, 10, 7} 

Iv 213 (9, 7, 10] 

Vv 23) (10, 7, 9] 

VI 321 (10, 9, 7] 


Figure 7.3: Sample input permutations 


case computing time of Q(r log 2). 


Proof: We must show that for every decision tree with 1! leaves, there is a path 
of length cnlogzn, where ¢ is a constant. By the theorem, there is a path of 
Tength logan !. Now 


nt=n(a— 12-2) +++ BX2\1)2@2)"" 
So, logan ! > (n/2)log)(n/2) = Q(nlog n).0 
Using a similar argument and the fact that binary trees with 2" leaves must 
have an average root-to-leaf path length of Q(n log n), we can show that the 
average complexity of comparison-based sorting methods is Q(n log n). 
7.55 MERGE SORT 
7.5.1 Merging 


Before looking at the Merge Sort method to sort » records, let us see how one 
may merge two sorted lists to get a single sorted list. We shall examine two 
different algorithms. The first one, Program 7.7, is very simple and uses O(n) 
additional space. The two lists to be merged are initList(l:m] and 
inisList [m +1:n}, The resulting merged list is mergedList [!:n]. 


Analysis of Merge: At each iteration of the for loop, éResult increases by 1. 
The total increment in iResult is at most n - 1 +}. Hence, the for toop is iterated 
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template <class T> 
void Merge(T *initList, T *mergedList, const int /, const int m, const int 1) 
{iH initList (t:m) and initList [m + lin] are sorted lists. They are merged to obtain 
Hf the sorted list mergedList [I:n]. 
for (int it =/, Result =1,i2=m+1; / il, i2, and iResuit are list positions 
i <= m && i2 <= n; // neither input list is exhausted 
iResult++) 


if (initList (i1] <= initList {i2]) 
{ 


mergedList (iResult | = inirList [il] 


ih++; 

} 

else 
mergedList {iResult ) = initList [i2); 
i244; 

} 


4 copy remaining records, if any, of first list 
copy (initList + il, initList +m + 1, mergedList + iResult); 


i copy remaining records, if any, of second list 
copy (initList + i2, initList +n + 1, mergedList + iResult); 


} 


— 


Program 7.7: Merging two sorted lists 


at most 1-1 +1 times. The copy statements copy at most a-/+1 records. 
The total time is therefore O(7 —/ + 1). . 

If each record has a size s, then the time is O(s(n —/ + 1). When s is 
greater than 1, we can use linked lists instead of arrays and obtain a new sorted 
linked list containing these n -/ + | records. Now, we will not need the addi- 
tional space for n —1+ 1 records as needed in Merge for the array mergedList. 
Instead, space for n-/ +1 links is needed. The merge time becomes 
O(n =1+ 1) and is independent of s, Note that n-/ +1 is the number of 
records being merged. O 
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7.5.2 Iterative Merge Sort 


This version of Merge Sort begins by interpreting the input list as comprised of n 
sorted sublists, each of size t. In the first merge pass, these sublists are merged 
by pairs to obtain 1/2 sublists, each of size 2 (if n is odd, then one sublist is of 
size 1). In the second merge pass, these 2/2 sublists are then merged by pairs to 
obtain n/ sublists. Each merge pass reduces the number of sublists by half. 
Merge passes are continued until we are left with only one sublist. The example 
below illustrates the process. 


Example 7.5: The input list is (26, 5, 77, 1, 61, 11, 59, 15, 48, 19). The tree of 
Figure 7.4 illustrates the sublists being merged at each pass. O 


Figure 7.4: Merge tree 


__ Since a Merge Sort consists of several merge passes, it is convenient first to 
write a function (Program 7.8) for a merge pass. Now the sort can be done by 
Tepeatedly invoking the merge-pass function as in Program 7.9. 


Analysis of MergeSort: Function MergeSort makes several passes over the 
records being sorted. In the first pass, lists of size | are merged. In the second, 
the size of the lists being merged is 2. On the ith pass the lists being merged are 
of size oh. Consequently, a total of [log, n] passes are made over the data. 
Since two lists can be merged in linear time (function Merge), each pass of 
Merge Sort takes O(2) time. The total computing time is O(nlogn). 0 
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template <class T> 
void MergePass(T *initList, T *resultList, const int n, const int s) 
{/ Adjacent pairs of sublists of size s are merged from 
H initList to resultList. n is the number of records in initList. 
for (int i = 1; // iis first position in first of the sublists being merged 
i<=n-—2+s+ 1; / enough elements for two sublists of length s? 
i+= 245) 
Merge(initList, resultList,i,i + s—1,i+2*s5~1)5 


H merge remaining list of size <2 * 5 
if (i + s - 1) <n) Merge(initList, resultList, i, i + s— 1, n)3 
else copy (initList + i, initList +n + 1, resultList + i); 


} 


Program 7.8: Merge pass 


eee 


template <class T> 

void MergeSort(T *a, const int n) 

{// Sort a(1:n] into nondecreasing order. 
T *tempList = new T[n +1]; 
// Lis the length of the sublist currently being merged 
for (int! = 1; /<n; 1 *= 2) 


MergePass(a, tempList, n, 1); 
Is= 2; . 
MergePass(tempList, a,n, 1); // interchange role of a and tempList 


delete [ ] renpList; 


Program 7.9: Merge Sort 
You may verify that MergeSort is a stable sorting function. 
7.5.3 Recursive Merge Sort 


In the recursive formulation we divide the list to be sorted into two roughly eal 
parts called the left and the right sublists. These sublists are sorted recursively, 
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and the sorted sublists are merged. 


Example 7.6: The input list (26, 5, 77, 1, 61, 11, 59, 15, 49, 19) is to be sorted 
using the recursive formulation of Merge Sort. If the sublist from Jeft to right is 
currently to be sorted, then its two sublists are indexed from left to 
[left + righty2] and from |(left + right2] + 1 to right. The sublist partition- 
ing that takes place is described by the binary tree of Figure 7.5. Note that the 
sublists being merged are different from those being merged in MergeSort. 0 


Figure 7.5: Sublist partitioning for Recursive Merge Sort 


To eliminate the record copying that takes place when Merge (Program 
7.7) is used to merge sorted sublists we associate an integer pointer with each 
record. For this purpose, we employ an integer array fink [1:n] such that link [i] 
gives the record that follows record i in the sorted sublist. In case link (#] =90, 
there is no next record. With the addition of this array of links, record copying is 
teplaced by link changes and the runtime of our sort function becomes indepen- 
dent of the size s of a record. Also the additional space required is O(n). By com- 
Parison, the Iterative Merge Sort described earlier takes O(snlogn) time and 
O(sn) additional space. On the down side, the use of an array of links yields a 
sorted chain of records and we must have a follow up process to physically rear- 
range the records into the sorted order dictated by the final chain. We describe 
the algorithm for this physical rearrangement in Section 7.8. 

We assume that initially fink [i] = 0, 1 < isn. Thus, each record is initially 
in a chain containing only itself. Let start and start2 be pointers to two chains 
of records. The records on each chain are in nondecreasing order. Let 
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ListMerge (a, link, start], start2) be a function that merges two chains start! 
and start2 in array a and returns the first position of the resulting chain that is 
linked in nondecreasing order of key values. The recursive version of Merge 
Sort is given by function rMergeSort (Program 7.10). To sort the array a{I:n] 
this function is invoked as rMergeSort (a , link, 1, n). The start of the chain 


ordered as described earlier is returned. Function ListMerge is given in Program 
WA, 


template <class T> 
int rMergeSort(T* a, int* link, const int left, const int right) 
{i a [left :right] is to be sorted. fink [i] is initially 0 for all i. 
H rMergeSort returns the index of the first element in the sorted chain. 
if (left >= right) return left; 
int mid = (left + right) / 2; 
return ListMerge(a, link, 
rMergeSort(a, link, left, mid), 4 sort left half 


rMergeSort(a, link, mid + 1, right)); # sort right half 
} 


Program 7.10: Recursive Merge Sort 


Analysis of rMergeSort: It is easy to see that the recursive Merge Sort is stable, 
and its computing time is O(n log n). O 


Variation—Natural Merge Sort: We may modify MergeSort to take into 
account the prevailing order within the input list. In this implementation we 
make an initial pass over the data to determine the sublists of records that are in 
order. Merge Sort then uses these initially ordered sublists for the remainder of 
the passes. Figure 7.6 shows Natural Merge Sort using the input sequence of 
Example 7.6. 


EXERCISES 
1. Write the status of the list (12, 2, 16, 30, 8, 28, 4, 10, 20, 6, 18) at the end of 
each phase of MergeSort (Program 7.9). 


2. Suppose we use Program 7.12 to obtain a Merge Sort function. Is the 
resulting function a stable sort? 


3. Prove that MergeSort is stable. 
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template <class T> 
int ListMerge(T* a, int* link, const int start, const int start2) 
{// The sorted chains beginning at start| and start2, respectively, are merged. 
i tink (0] is used as a temporary header. Retum start of merged chain. 
int iResult=0; // last record of result chain 
for (int il = start, i2 = start2; i] && i2;) 
if (a (i1] <= a {i2]) { 
link [iResult] = i]; 
iResult = il; i = link {i}; 


else { 
link [iResult ] = i2; 
iResult = i2} i2 = link (12); 
} 


4 attach remaining records to result chain 
if (1 == 0) link [Result } = i2; 
else link [iResult} = i1; 

} return link {0}; 


Program 7.11: Merging sorted chains 


26] [5 7 1 61} {ar 59} [15 48) L19] 


5 26 77 1 lL 5961] [15 19 48 


Figure 7.6: Natural Merge Sort 
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4. Write an iterative Natural Merge Sort function using arrays as in function 
MergeSort. How much time does this function take on an initially sorted 
list? Note that MergeSort takes O(t log 2) on such an input list. What is 
the worst-case computing time of the new function? How much additional 
space is needed? 


5. Do the previous exercise using chains. 


7.6 HEAPSORT 


Although the Merge Sort scheme discussed in the previous section has a comput- 
ing time of O(n log ), both in the worst case and as average behavior, it requires 
additional storage proportional to the number of records to be sorted. By using 
the O(1) space merge algorithm, the space requirements can be reduced to O(1). 
The resulting sort algorithm is, however, much slower than the original one. The 
sorting method we are about to study, Heap Sort, requires only a fixed amount of 
additional storage and at the same time has as its worst-case and average com- 
puting time O(n log n). Although Heap Sort is slightly slower than Merge Sort 
using O(n) additional space, it is faster than Merge Sort using O(1) additional 
space. 

In Heap Sort, we utilize the max-heap structure introduced in Chapter 5. 
The deletion and insertion functions associated with max heaps directly yield an 
O(n Jog n) sorting method. The 1 records are first inserted into an initially empty 
max heap. Next, the records are extracted from the max heap one at a time. Itis 
possible to create the max heap of n records faster than by inserting the records 
one by one into an initially empty heap. For this, we use the function Adjust (Pro- 
gram 7.13), which starts with a binary tree whose left and right subtrees are max 
heaps and rearranges records so that the entire binary tree is a max heap. The 
binary tree is embedded within an array using the standard mapping. If the depth 
of the tree is d, then the for loop is executed at most d times. Hence the comput- 
ing time of Adjust is O(d). . 

To sort the list, first we create a max heap by using Adjust repeatedly, as in 
the first for loop of function HeapSort (Program 7.14). Next, we swap the first 
and last records in the heap. Since the first record has the maximum key, the 
swap moves the record with maximum key into its correct position in the sorted 
array. We then decrement the heap size and readjust the heap. This swap, decre- 
ment heap size, readjust heap process is repeated n — | times to sort the entire 
array a(1:n]. Each repetition of the process is called a pass. For example, on 
the first pass, we place the record with the highest key in the nth position, on the 
second pass, we place the record with the second highest key in position n-k 
and on the ith pass, we place the record with the ith highest key in position #—/ 
+1. The invocation is HeapSorta, n). 
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template <class T> 
void Adjust(T *a, const int root, const int n) : 
{// Adjust binary tree with root root to satisfy heap property. The left and right 
4 subtrees of root already satisfy the heap property. No node index is > a. 
Te =al[roor); 
# find proper place for e 
for (int j = 2*root; j <= nj; j #=2){ : . 
GU <n &&alj}<alj+1) jt; jis max child of its parent 
if (e >= a{j]) break; // e may be inserted as parent of j 
a[j/2] =a[j); 4 move jth record up the wee 


alj/2j=e; 


Program 7.13: Adjusting a max heap 


template <class T> 
void HeapSort(T *a, const int n) 
{/ Sort a[1:n }) into nondecreasing order. 
for (int i =n /2; i >= 1; i-—) // heapify 
Adjust(a, i, n); 


for (i=n-1;i >= 1;i--) M sort 


swap (a[]), afi +1))5 // swap first and last of current heap 
Adjust(a, 1, i); MU heapify 


Program 7.14: Heap Sort 


Example 7.7: The input list is (26, 5, 77, 1, 61, 11, 59, 15, 48, 19). If we inter- 
pret this list as a binary tree, we get the tree of Figure 7.7(a). Figure 7.7(b) dep- 
icts the max heap after the first for loop of HeapSort. Figure 7.8 shows the array 
of records following each of the first seven iterations of the second for loop. The 
Portion of the array that still represents a max heap is shown as a binary tree; the 
sorted part of the array is shown as an array. O 


Analysis of HeapSort: Suppose 2! <n < 24, so the tree has k levels and the 
number of nodes on level i is < 2'-'. In the first for loop, Adjust (Program 7.13) 
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(8) [9] [10] [8] [9} (10) 
(a) Input array (b) Initial heap 


Figure 7.7: Array interpreted as a binary tree 


is called once for each node that has a child. Hence, the time required for this 
loop is the sum, over each level, of the number of nodes on a level multiplied by 
the maximum distance the node can move. This is no more than 


LA ke)e YD Wi <n Y isd! < n=O) 
{Sisk ISisk-1 Isisk-1 


In the next for loop, n — 1 applications of Adjust are made with maximum wee- 
depth k = [log, (n + 1)] and swap is invoked n — 1 times. Hence, the comput- 
ing time for this loop is O(n log). Consequently, the total computing time 1s 
O(n logn). Note that apart from some simple variables, the only additional 
space needed is space for one record to carry out the swap in the second for 
loop. 0 


EXERCISES 


I. Write the status of the list (12, 2, 16, 30, 8, 28, 4, 10, 20, 6, 18) at the end of 
the first for loop as well as at the end of each iteration of the second for 
loop of HeapSort (Program 7.14). 

2. Heap Sort is unstable. Give an example of an input list in which the order 
of records with equal keys is not preserved. 
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(7 


(a) Heap size = 9 (b) Heap size = 8 
Sorted = [77} Sorted = [61, 77] 


[6] 7 {6} 
(c) Heap size = 7 (d) Heap size =6 
Sorted = [59, 61, 77] Sorted = [48, 59, 61, 77] 


Figure 7.8: Heap Sort example (continued on next page) 


7.7 SORTING ON SEVERAL KEYS 


We now look at the problem of sorting records on several keys, K', K?, -+-, K" 
(K' is the most significant key and K* the least). Fe Lise of records R,. “Ry i is 
said to be sorted with Tespect to the keys K), K?, , K° iff for every pale of 
records i and j, i < j and (K}, -+-, K‘) <(Ki, ¢ Ki). The r-tuple (x), °°", 
,) is less than or equal to the ‘tuple Onc Ye) iff either xp =¥, 1 SiSj. and 
X41 <Yj41 for some j <r, or x; = yj, 1Si Sr. 

For example, the problem of sorting a deck of cards may be regarded as a 
sort on two keys, the suit and face values, with the following ordering relations: 
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(119) a) iv) 
Kis) BK) RKS BK) G) (1) 


(2} (3) 
3) 
4) [5] (4) 
(e) Heap size = 5 (f) Heap size = 4 (g) Heap size =3 


[26, 48, 59, 61,77) (19, 26, 48, 59, 61, 77)[15, 19, 26, 48, 59, 61, 77] 


Figure 7.8: Heap Sort example 


K' (Suits): acecuca 
K? [Face values}; 2<3<4--- <10<J<Q<K<A 


A sorted deck of cards therefore has the following ordering: 
2m... Ady... 20,62, Am 


‘There are two popular ways to sort on multiple keys. In the first, we begin 
by sorting on the most significant key K!, obtaining several ‘‘piles’’ of records, 
each having the same value for K'. Then each of these piles is independently 
sorted on the key K? into *‘subpiles’’ such that all the records in the same sub- 
pile have the same values for K! and K?. The subpiles are then sorted on K”, 
and so on, and the piles are combined. Using this method on our card deck 
example, we would first sort the 52 cards into four piles, one for each of the suit 
values, then sort each pile on the face value. Then we would place the piles on 
top of each other to obtain the desired ordering. A aa 

A sort proceeding in this fashion is referred to as a most-significant-digit- 
first (MSD) sort. The second way, quite naturally, is to sort on the least 
significant digit first (LSD). An LSD sort would mean sorting the cards first into 
13 piles corresponding to their face values (key K 2). Then, we would place the 
3's on top of the 2’s, «+ ~, the kings on top of the queens, the aces on top of the 
kings; we would turn the deck upside down and sort on the suit (K") using & 
stable sorting method to obtain four piles, each orderd on K?; and we would 
combine the pites to obtain the required ordering on the cards. 

Comparing the two functions outlined here (MSD and LSD), we see that 
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LSD is simpler, as the piles and subpiles obtained do not have to be sorted 
independently (provided the sorting scheme used for sorting on the keys K‘, | 
Si <r, is stable). This in turn implies less overhead. 

The terms LSD and MSD specify only the order in which the keys are to be 
sorted. They do not specify how each key is to be sorted. When sorting a card 
deck manually, we generally use an MSD sort. The sorting on suit is done by @ 
bin sort (i.e., four **bins’’ are set up, one for each suit value and the cards are 
placed into their corresponding bins). Next, the cards in each bin are sorted 
using an algorithm similar to Insertion Sort. However, there is another way to do 
this. First use a bin sort on the face value. To do this we need 13 bins, one for 
each distinct face value. Then collect all the cards together as described above 
and perform a bin sort on the suits using four bins. Note that a bin sort requires 
only O(n) time if the spread in key values is O(). 

LSD or MSD sorting can be used to sort even when the records have only 
one key. For this, we interpret the key as being composed of several subkeys. 
For example, if the keys are numeric, then each decimal digit may be regarded as 
a subkey. So, if the keys are in the range 0 < K < 999, we can use either the LSD 
or MSD sorts for three keys (K', K?, K?), where K! is the digit in the hun- 
dredths place, K? the digit in the tens place, and K° the digit in the units place. 
Since 0 < K‘ <9 for each key K', the sort on each key can be carried out using a 
bin sort with 10 bins. 

in a Radix Sort, we decompose the sort key using some radix r, When r is 
10, we get the decimal decomposition described above. When r = 2, we get 
binary decomposition of the keys. In a Radix-r Sort, the number of bins required 
is r. 

Assume that the records to be sorted are R,, *--,R,. The record keys are 
decomposed using a radix of r. This results in each key having d digits in the 
range 0 through r — 1. Thus, we shall need r bins. The records in each bin will 
be linked together into a chain with [i], 0 <i <r, a pointer to the first record in 
bin i and efi],a pointer to the last record in bin i, These chains will operate as 
pla Function RadixSort (Program 7.15) formally presents the LSD radix-r 
method. 


Analysis of RadixSort: RadixSort makes d passes over the data, each pass tak- 
ing O(u + r) time. Hence, the total computing time is O(d(n + r)). The value of 
d will depend on the choice of the radix r and also on the largest key. Different 
choices of r will yield different computing times. 0 


Example 7.8: Suppose we are to sort 10 numbers in the range (0, 999]. For this 
example, we use r = 10 (though other choices are possible). Hence, d = 3. The 
input list is linked and has the form given in Figure 7.9(a). The nodes are labeled 
Ry, +++, Ri. Figure 7.9 shows the queues formed when sorting on each of the 
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template <class T> 
int RadixSort(T *a, int *link, const int d, const int r, const int n) 
{# Sort a[1:n}) using a d-digit radix-r sort. digit (a [i},j,r) retums the jth radix-r 
# digit (from the left) of a [7]'s key. Each digit is in the range is [0,r). 
# Sorting within a digit is done using a bin sort. 
int e(r}, f (J; // queue front and end pointers 
H create initial chain of records starting at first 
int first = 1; 
for (int i= 1; i < nj i++) link [i] =i + 1; // link into a chain 
link {n] = 03 


for (i=d - 1; §>=0;i--) 
{#/ sort on digit 
Sill, f + r, 0); initialize bins to empty queues 
for (int current = first; current; current = link (current }) 
{/ put records into queues/bins 
int k = digit (a {current}, i, r); 
if (f [k = current; 
else link [e [k]] = current; 
e[k] =current; 


} 
for (j = 05 !f [i]; j ++); / find first nonempty queue/bin 
first =f Ui 
int last = e Lj); 
for (int k= j + 1; k <r; k++) / concatenate remaining queues 
iff) 
link {last ] = f (kK); 
last = e[k]; 


} 
link [last } = 03 
} 
return first; 


) 


Program 7.15: LSD Radix Sort 


digits, as well as the lists after the queues have been collected from the 10 bins. 
o 
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a(t} af2] a[3] al[4j a[5] a[6] af7] a@f8) a(9} flo} 


[179 } {208 | +{306} »[ 93 } »{859} »[984] of $5 9} +271} +f 33 


(a) Initial input 

el) eft] (2) ef3} ld] lS) el] al] altel) 
[>] 

33} (859} 

aH 384] [35] (303) (08) (179 


Ft) FIN fl2) 03) F451 16 FIT) F181 #19) 


[27T} [93] +133 } of 984} +55} 1506} [208] =f179) = [859] [7] 


(b) First-pass queues and resulting chain 


ef0] efl) ef2] e{3} ef4] ef5} ef6) ef7] [8] [9] 


ca | | 

208 859 179 

306 33} 35] 271} [984] [93 
q x 


ate 
ff) F218) Fa) 15) 


[306]+[208] {9 } +{ 33} {55 }->{859}-={271] »{179} + 984] =f 93 


(c} Second-pass queues and resulting chain 


Figure 7.9: Radix Sort example (continued on next page) 
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e{1] e(2] [3] el4]  efS]_ ef6}_ ef7]ef8)_— a9} 


271 


i 
179] [208] [306 [859] 


JO} ft) 2} FB] FU) FES] ff) F718] 


[TPE 55] 97} =] =] +27 306} ~ 555} [958] 


(d) Third-pass queues and resulting chain 


Figure 7.9: Radix Sort example 


EXERCISES 


f. 


2. 


Write the status of the list (12, 2, 16, 30, 8, 28, 4, 10, 20, 6, 18) at the end of 
each pass of RadixSort (Program 7.15). Use r = 10. 


Under what conditions would an MSD Radix Sort be more efficient than an 
LSD Radix Sort? 


Does RadixSort result in a stable sort when used to sort numbers as In 
Example 7.8? 

Write a sort function to sort records Ry, °**,R, lexically on keys 
(K', -+», K") for the case when the range of each key is much larger than 
n. In this case, the bin-sort scheme used in RadixSort to sort within each 
key becomes inefficient (why?). What scheme would you use fo sort within 
a key if we desired a function with (a) good worst-case behavior, (b) good 
average behavior, (c) small n, say <15? 

If we have n records with integer keys in the range 0, n7), then they may 
be sorted in O( log n) time using Heap Sort or Merge Sort. Radix Sort on 
a single key (ie.,d = land r= n?) takes O(n?) time. Show how to inter- 
pret the keys as two subkeys so that Radix Sort will take only Of) time to 
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sort n records. (Hint: Each key, K;, may be written as K, = K}n + K? with 
K} and K? integers in the range {0, ).) 

6. Generalize the method of the previous exercise to the case of integer keys 
in the range (0, 2”) obtaining an O(pz) sorting method. 

7. Experiment with RadixSort to see how it performs relative to the 
comparison-based sort methods discussed in earlier sections. 
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Apart from Radix Sort and Recursive Merge Sort, all the sorting methods we 
have looked at require excessive data movement. That is, as the result of a com- 
parison, records may be physically moved. This tends to slow down the sorting 
Process when records are large. When sorting lists with large records, it is 
necessary to modify the sorting methods so as to minimize data movement. 
Methods such as Insertion Sort and our Iterative Merge Sort can be modified to 
work with a linked list rather than a sequential list. In this case each record will 
require an additional link field. Instead of physically moving the record, we 
change its link field to reflect the change in the position of the record in the list. 
At the end of the sorting process, the records are linked together in the required 
order. In many applications (e.g., when we just want to sort lists and then output 
them record by record on some extemal media in the sorted order), this is 
sufficient. However, in some applications it is necessary to physically rearrange 
the records in place so that they are in the required order. Even in such cases, 
considerable savings can be achieved by first performing a linked-list sort and 
then physically rearranging the records according to the order specified in the 
list. This rearranging can be accomplished in linear time using some additional 
space. 

If the list has been sorted so that at the end of the sort, first is a pointer to 
the first record in a linked list of records, then each record in this list will have a 
key that is greater than or equal to the key of the previous record (if there is a 
previous record). To physically rearrange these records into the order specified 
by the list, we begin by interchanging records R and Rérs. Now, the record in 
the position Ry has the smallest key. If first # 1, then there is some record in the 
list whose link field is 1, If we could change this link field to indicate the new 
position of the record previously at position 1, then we would be left with 
records R2, ---, R, linked together in nondecreasing order. Repeating the 
above process will, after 7 — 1 iterations, result in the desired rearrangement. 
The snag, however, is that in a singly linked list we do not know the predecessor 
of a node. To overcome this difficulty, our first rearrangement function, List] 
(Program 7.16), begins by converting the singly linked list first into a doubly 
linked list and then proceeds to move records into their correct places. This 
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function assumes that links are stored in an integer array as in the case of our 
Radix Sort and Recursive Merge Sort functions. 


template <class T> 
void Listl(T *a, int *linka, const int n, int first) 
{// Rearrange the sorted chain beginning al first so that the records a [1:71] 
# are in sorted order. 
int */inkb = new int [n]; // array for backward links 
int prev = 0; 
for (int current = first; current; current = linka [current ]) 
{/# convert chain into a doubly linked list 
linkb (current ) = prev; 
prev = current; 


} 


for (int i= 1; i<n; i++) / move a [first] to position i while 
# maintaining the list 
if (frst '= i) { 


if (linka {i }) linkb [linka [i] = first; 
linka [linkb [i]] = first; 

swap (a[first], afi); 

swap (linka [first], linka [i)}); 

swap (linkb [first |, linkb [i )); 


} 
first = linka (i); 


j 


Program 7.16: Rearranging records using a doubly linked list 


Example 7.9: After a List Sort on the input list (26, 5, 77, 1, 61, 11, 59. 15, 48, 
19) has been made, the list is linked as in Figure 7.10(a) (only the key and link 
fields of each record are shown). Following the links starting at first, we obtain 
the logical sequence of records Ry, Rr, Ro. Rg, Ryo, Ri, Ro, Ra Rs, and ae 
This sequence corresponds to the key sequence 1, 5, 11, 15, 19, 26, 48, 59, ay 
33. Filling in the backward links, we get the doubly linked list of Figure rie ). 
Figure 7.11 shows the list following the first four iterations of the second for 
loop of List1. The changes made in each iteration are shown in boldface. Qo 


Analysis of List!: If there are 1 records in the list, then the time required to me 
vert the chain first into a doubly linked fist is O(n). The second for loop ! 
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i R, | Ro | Rs | Ra | Rs | Re | Rp | Re | Ro LR 
key | 26] 5|7| 1{ 61] 11; 59] 15 | 48] 19 
linka |! 9[ 6[ Of 2] 3{ 8| S{ 1} 7] 1 


(a) Linked list following a List Sort, first = 4 


i TR, [Rp] Rs | Ra Ry 7 
key | 26| 5) 77] 1 48 {_ 19 
linka | 91 6[ Of 2 7/1 
links | 10; 4/ 5/0 1 8 


(b) Corresponding doubly linked list, first = 4 


Figure 7.10: Sorted linked lists 


iterated n - | times. In each iteration, at most two records are interchanged. This 
Tequires three record moves. If each record is m words long, then the cost per 
record swap is O(m). The total time is therefore O(tun). 

The worst case of 3(n — 1) record moves (note that each swap requires 3 
record moves) is achievable. For example, consider the input key sequence Ry, 
Ro, +++, Ry, With Rp <R3< °++ <R, andR, >R,. 0 


Although several modifications to Listl are possible, one of particular 
interest was given by M. D. MacLaren. This modification results in a rearrange- 
ment function, List2, in which no additional link fields are necessary. In this 
function (Program 7.17), after the record Rérs: is swapped with R;, the link field 
of the new R; is set to first to indicate that the original record was moved. This, 
together with the observation that first must always be 2i, permits a correct 
reordering of the records. 


Example 7.10: The data is the same as in Example 7.9. After the List Sort we 
have the configuration of Figure 7.10(a). The configuration after each of the first 
five iterations of the for loop of List2 is shown in Figure 7.12. 0 


Analysis of list2: The sequence of record moves for List2 is identical to that for 
fist!. Hence, in the worst case 3(n - 1) record moves for a total cost of O(nm) 
are made. No node is examined more than once in the while loop. So, the total 
time for the while loop is O(n). G 
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i 1 | R2 [Rs | Ra [Rs [Re | Rr | Re | Ro | Reo 
key 1 5 | 77: 2%] 61 it 59 15 | 48 19 
linka | 2{ 6{ Of 9] 3] 8[ 5] lO[ 7] 4 
linkb 0 4 | 5 10 7} 2 9 6 | 4 8 


(a) Configuration after first iteration of the for loop of Listl, first = 2 


i Ry | Ry | R3 | Ry | Rs 


1 Re | R71 | Ro | Ro | Rio | 
1] 5‘ 77 2%] 61 | {| so| is[ 48] 19 
linka | 21 6] O| 9 3[ 8[ S|] iol 7] 4 
Tinks | Of[ 4] 5j fo[ 7) 2] 9[ 6] 4 8 


i 


he~ feb 

(key. | 0 35 
| linka 2 6 
[linko | OT 4 


(d) Configuration after fourth iteration, first = 10 


Figure 7.11: Example for List] (Program 7.16) 


Although the asymptotic computing time for both List! and List2 is the 
same, and the same number of record moves is made in either case, We | 
List2 to be slightly faster than List] because each time two records are swapped, 
List\ does more work than Lisr2 does. Listl is inferior to List in both space and 
time considerations. 

The List Sort technique is not well suited for Quick Sort and Heap Sort. 
The sequential representation of the heap is essential to Heap Sort. For these 
sort methods, as well as for methods suited to List Sort, one can maintain 4A 
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template <class T> 
void List2(T *a, int */ink, const int n, int first) 
{/ Same function as List! except that a second link array linkb is not required. 
for (int i = 1; i <n; i++) 
{4 Find correct record for ith position. Its index is 2 i as 
H records in positions 1,2, ---,i— 1 are already correctly positioned. 
while (first < i) first = link [first]; 
int q = link [first]; 4 a[q] is next record in sorted order 
if (first =i) 
{// a [first) has ith smallest key. Move record to ith position, 
4 Also set link from old position of a [i] to new one. 
swap (a [i], a [first}); 
link [first] = link [i]; 
link [i] = first; 


first = 95 
} 
Program 7.17; Rearranging records using only one link field 


auxiliary table, 1, with one entry per record. The entries in this table serve as an 
indirect reference to the records. 

At the start of the sort, [i] = i, 1 Sin. If the sorting function requires a 
swap of a[i] and a{j], then only the table entries (ie., t[/] and ¢[j]) need to be 
swapped. At the end of the sort, the record with the smallest key is a(¢{1]] and 
that with the largest a [r(7]]. The required permutation on the records is a(t [1]}, 
a(t(2]], ---, a[t{n]) (see Figure 7.13). This table is adequate even in situations 
such as binary search, where a sequentially ordered list is needed. In other situa- 
tions, it may be necessary to physically rearrange the records according to the 
permutation specified by 1. 

The function to rearrange records corresponding to the permutation #[1}, 
#(2], -->, ¢{m] is a rather interesting application of a theorem from mathemati 
Every permutation is made up of disjoint cycles, The cycle for any element i is 
made up of i, (i) 71), -++. 4), where (i) = rte}, el} =i, and 
t'{i] =i Thus, the permutation r of Figure 7.13 has two cycles, the first involv- 
ing R, and Rs and the second involving Ry, R3, and Rz. Function Table (Pro- 
gram 7.28) utilizes this cyclic decomposition of a permutation. First, the cycle 
containing R, is followed, and all records are moved to their correct positions. 
The cycle containing R, is the next one examined unless this cycle has already 
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i TR: | R [ R3 | 


Re | R7 | Re | Ro | Roo 
| key 1 5 a 


[59 | 15 | 48] 19 
a[ 3] ml 7{ 1 


(a) Configuration after first iteration of the for loop of Lis#2, first =2 


link 4 6 0 


i Ry | Ry Rs | Ra | Rs | Re | Rr | Re | Ro | Rw] 
| key 1] 5: 7] 26] 61 { 11 { 59] 15 | 48 | 19 
link 4] 6] O[ 9] 3[ 8] S| 10[ 7 i 


é [Ry Ry [Rs | Ra | Rs TR 
Ca 
link | 47 6] 6] 9] 3] 0] 


(e) Configuration after fifth iteration, first = 1 


Figure 7.12: Example for List2 (Program 7.17) 


been examined. The cycles for R3, R4, «°° 
The result is a physically sorted list. 

When processing a trivial cycle for R; (ie. t[/] = 8, 10 rearrangemen 
involving record R; is required, since the condition ¢[i} = i means that the reco! 
with the ith smallest key is R;. In processing a nontrivial cycle for record R ue 

1[i1 #3), R; is moved to a temporary position p, then the record at #[é} is move 
to i; next the record at ¢[r(é]} is moved to r{i], and so on until the end of the 
cycle t*{i] is reached and the record at p is moved to ¢*"" [i]. 


.R,-1 are followed in this order. 
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Auxiliary table ¢ before sorting 


Table 2 after sorting 
Figure 7.13: Table Sort 


template <class T> 
void Table(T *a, const int n, int +1) 
{4 Rearrange a[1:n] to correspond to the sequence a[?(1]], --*, a{t{n]}],n21. 
for (int i= 1; i< ns i++) 
if (¢(i] !=8) {/ there is a non-trivial cycle starting ati 
Tp=ali)s 
int j =i; 
do { 
int ‘ =U all =atkhtl=j 
Jak; 


} 


Program 7.18: Table Sort 
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Example 7.11; Suppose we start with the table 1 of Figure 7.14(a). This figure 
also shows the record keys. The table configuration is that following a Table 
Sort. There are two nontrivial cycles in the permutation specified by ¢. The first 
is R,,R3,Rg, RoR. The second is Ry, Rs,R7, Rg. During the first iteration (i 
= 1) of the for loop of sable (Program 7.18), the cycle Ry, Rey Ray Rey Ri 
is followed. Record Rj is moved to a temporary spot p; R,11} (1.€., R3) is moved 
to the position R 1; R,2(1) (i.€., Rg) is moved to R3; Re is moved to Rg; and finally 


p is moved to Rg. Thus, at the end of the first iteration we have the table 
configuration of Figure 7.14(b). 


key 


(a) Initial configuration 


[key | 12 | 4 [ie 7 42 [36 [35] 31 | 50) 
[yp [i [2 


(b) Configuration after rearrangement of first cycle 


key | 12] 147 18] 26] 31, 35 | 42 | 50 
ti [eilals4 is te [7 784 


(c) Configuration after rearrangement of second cycle 


Figure 7.14: Table Sort example 


For i = 2 or 3, s[i] =i, indicating that these records are already in their 
correct positions. When i = 4, the next nontrivial cycle is discovered, and the 
records on this cycle (R4, Rs, R7, R4) are moved to their correct positions. Fol- 
lowing this we have the table configuration of Figure 7.14(c). 

For the remaining values of i (i = 5, 6, and 7), [2] = é, and no more non- 
trivial cycles are found. G 


Analysis of Table: If each record uses m words of storage, then the additional 
space needed is m words for p plus a few more for variables such as i, j. and k. 
To obtain an estimate of the computing time, we observe that the for loop is exe- 
cuted n— 1 times. If for some value of i, [£] #4, then there is a nontrivial a 
including k > 1 distinct records R, Ry}, °° *+ Rey. Rearranging these reco 4 
requires k + 1 record moves. Following this, the records involved in this cycle 
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are not moved again at any time in the algorithm, as t[j] = j for all such records 
Rj. Hence, no record can be in two different nontrivial cycles. Let k; be the 
number of records on a nontrivial cycle starting at Ry when i = / in the for loop. 
Let k, = 0 fora trivial cycle. The total number of record moves is 

nol 

x k+l) 

150 kod 

Since the records on nontrivial cycles must be different, x; $n. The total 
number of record moves is maximum when YX, = n and there are |n/2| cycles. 
When n is even, each cycle contains two records. Otherwise, one cycle contains 
three and the others two each. In either case the number of record moves is 
[3n/2]. One record move costs O(m) time. The total computing time is there- 
fore O(mn). 0 


Comparing List2 (Program 7.17) and Table, we see that in the worst case, 
List2 makes 3(n—1) record moves, whereas Table makes only |3n/2] record 
moves. For larger values of m it is worthwhile to make one pass over the sorted 
list of records, creating a table ¢ corresponding to a Table Sort. This would take 
O(n) time. Then Table could be used to rearrange the records in the order 
specified by 1. 


EXERCISES 


1. Complete Example 7.9. 
2. Complete Example 7.10. 


3. Write a version of Selection Sort (see Chapter 1) that works on a chain of 
records. 

4. Writea Table Sort version of Quick Sort. Now during the sort, records are 
not physically moved. Instead, ¢[i} is the index of the record that would 
have been in position i if records were physically moved around as in 
QuickSort (Program 7.6). Begin with t[i] =i, 1<i<n. Atthe end of the 
sort, (i) is the index of the record that should be in the ith position in the 
sorted list. So now function Table may be used to rearrange the records 
into the sorted order specified by 1. Note that this reduces the amount of 
data movement taking place when compared to QuickSort for the case of 
large records. 

5. Do Exercise 4 for the case of Insertion Sort. 

6. Do Exercise 4 for the case of Merge Sort. 


7. Do Exercise 4 for the case of Heap Sort. 
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7.9 SUMMARY OF INTERNAL SORTING 


Of the several sorting methods we have studied, no one method is best under all 
circumstances. Some methods are good for small n, others for large n. Insertion 
Sort is good when the list is already partially ordered. Because of the low over- 
head of the method, it is also the best sorting method for *‘smail’’ n. Merge Sort 
has the best worst-case behavior but requires more storage than Heap Sort. 
Quick Sort has the best average behavior, but its worst-case behavior is O(n?). 
The behavior of Radix Sort depends on the size of the keys and the choice of r. 


Figure 7.15 summarizes the asymptotic complexity of the first four of these sort 
methods. 


Method Worst Average 
i 


Insertion Sort =n n 
Heap Sort nlogn nlogn 
Merge Sort nlogn nlogn 
Quick Sort n? nlogn 


ee 


Figure 7.15: Comparison of sort methods 


Figure 7.16 gives the average runtimes for the four sort methods of Figure 715. 
These times were obtained on a 1.7GHz Intel Pentium 4 PC with 512 MB RAM 
and Microsoft Visual Studio NET 2003. For each x at least 100 randomly gen- 
erated integer instances were run. These random instances were constructed by 
making repeated calls to the C++ function rand. If the time taken to sort these 
instances was less than } second then additional random instances were sorted 
until the total time taken was at least this much. The times reported in Figure 
7.16 include the time taken to set up the random data. For each n the time taken 
to set up the data and the time for the remaining overheads included in the 
reported numbers is the same for all sort methods. As a result, the data of Figure 
7.16 is useful for comparative purposes. The data of this figure is plotted in Fig- 
ure 7.17. 

As Figure 7.17 shows, Quick Sort outperforms the other sort methods for 
suitably large ». We see that the break-even point between Insertion and Quick 
Sort is between 50 and 100. The exact break-even point can be found expen- 
mentally by obtaining run-time data for n between 50 and 100. Let the exact 
break-even point be nBreak. For average performance, Insertion Sort is the best 
sort method (of those tested) to use when n < nBreak, and Quick Sort is the best 
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n Insert Heap Merge Quick 

0 0.000 =—-0.000 0.000 =: 0.000 

50 0.004 0.009 0.008 0.006 
100 0.01) 0.019 0.017 0.013 
200 0.033 0.042 0.037 0.029 
300 0.067 0.066 0.059 0.045 
400 0.117 0,090 0.079 = 0.061 
500 0.179 0.116 0.100 0,079 
1000 0.662 0.245 0.213 0.169 
2000 2.439 = 0.519 0.459 0.358 
3000 5.390 0.809 0.721 0.560 
4000 9.530 1.105 0.972 0.761 


5000 15.935 1.410 1.271 0.970 
Times are in milliseconds 


Figure 7.16: Average times for sort methods 


when n >nBreak. We can improve on the performance of Quick Sort for 
n > nBreak by combining Insertion and Quick Sort into a single sort function by 
replacing the following statement in Program 7.19 


if (left < right) { 
code to partition and make recursive calls 


with the code 


if (left + nBreak < right) { 
code to partition and make recursive calls 


else { 
sort a [left:right] using Insertion Sort; 
return; 


For worst-case behavior most implementations will show Merge Sort to be 
best for n > where ¢ is some constant. For m $c Insertion Sort has the best 
worst-case behavior. The performance of Merge Sort can be improved by com- 
bining Insertion Sort and Merge Sort in a manner similar to that described above 
for combining Insertion Sort and Quick Sort. 

The run-time results for the sort methods point out some of the limitations 


434 Sorting 


Se Insertion Sort 


4L 
3k 
fob 
Heap Sort 
Merge Sort 
tk 


Quick Sort 


Figure 7.17: Plot of average times (milliseconds) 


of asymptotic complexity analysis. Asymptotic analysis is not a good predictor 
of performance for smal! instances—insertion sort with its O(n?) complexity is 
better than all of the O(nlogn) methods for small instances. Programs that have 
the same asymptotic complexity often have different actual runtimes. 


C++’s Sort Methods 

If you had to develop the STL sort function sort, what would you do? Should 
you pick a method that optimizes worst-case performance, or should you pick 
one that optimizes average performance? Should you limit your choice to a sort 
method that is stable (i.c., a method that does not alter the relative order of equal 
elements)? 

The developers of C++'s sort function opted to optimize average perfor- 
mance and use a modified Quick Sort that reverts to a Heap Sort when the 
number of subdivisions exceeds some constant times logzr and to an Insertion 
Sort when the segment size becomes small. The STL function stable_sort is a 
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Merge Sort that reverts to an Insertion Sort when the segment size becomes 
small. The STL function partial_sort is based on Heap Sort and has the ability to 
stop when only the first k elements need to be sorted. 


EXERCISES 


1. 


{Count Sort] The simplest known sorting method arises from the observa- 
tion that the position of a record in a sorted list depends on the number of 
records with smaller keys. Associated with each record there is a count 
field used to determine the number of records that must precede this one in 
the sorted list. Write a function to determine the count of each record in an 
unordered list. Show that if the list has n records, then all the counts can be 
determined by making at most 1 (n — 1)/2 key comparisons. 


Write a function similar to Table (Program 7.18) to rearrange the records of 
a list if, with each record, we have a count of the number of records preced- 
ing it in the sorted list (see Exercise 1). 

Obtain Figures 7.18 and 7.19 for the worst-case runtime. 

[Programming Project] The objective of this assignment is to come up 
with a composite sorting function that is good on the worst-time criterion. 
The candidate sort methods are (a) Insertion Sort, (b) Quick Sort, (c) Merge 
Sort, (d) Heap Sort. 


To begin with, program these sort methods in C++. In each case, 
assume that n integers are to be sorted. In the case of Quick Sort, use the 
median-of-three method. In the case of Merge Sort, use the iterative 
method (as a separate exercise, you might wish to compare the runtimes of 
the iterative and recursive versions of Merge Sort and determine what the 
recursion penalty is in your favorite language using your favorite com- 
piler). Check out the correctness of the programs using some test data. 
Since quite detailed and working functions are given in the book, this part 
of the assignment should take little effort. In any case, no points are eamed 
until after this step. 

To obtain reasonably accurate runtimes, you need to know the accu- 
tacy of the clock or timer you are using. Determine this by reading the 
appropriate manual. Let the clock accuracy be 5. Now, mun a pilot test to 
determine ballpark times for your four sorting functions for n = 500, 1000, 
2000, 3000, 4000, and 5000. You will notice times of 0 for many of these 
values of n. The other times may not be much larger than the clock accu- 
racy. 

To time an event that is smaller than or near the clock accuracy, 
repeat it many times and divide the overall time by the number of repeti- 
tions. You should obtain times that are accurate to within 1%. 
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We need worst-case data for each of the four sort methods. The 
worst-case data for Insertion Sort are easy to generate. Just use the 
sequence n, n-1, n-2, ---, 1. Worst-case data for Merge Sort can be 
obtained by working backward. Begin with the last merge your function 
will perform and make this work hardest. Then look at the second-to-last 
merge, and so on. Use this logic to obtain a program that will generate 
worst-case data for Merge Sort for each of the above values of n. 

Generating worst-case data for Heap Sort is the hardest, so, here we 
shall use a random permutation generator (one is provided in Program 
7.20). We shall generate random permutations of the desired size, clock 
Heap Sort on each of these, and use the max of these times to approximate 
to the worst-case time. You will be able to use more random permutations 
for smaller values of n than for larger. For no value of » should you use 
fewer than 10 permutations. Use the same technique to obtain worst-case 
times for Quick Sort. 


SSSSSSSSSSSSSSeSSSSsFFSSsSFSSSSSSSSSseSSSSSSSeeeee 


template <class T> 
void Permute(T +a, int n) 
{// Random permutation generator. 
for (int i = n; i >= 2; i-~) 
{ 
int j = rand() % i + 1; / j = random integer in the range [1, é] 
swap (alj], ati); 


} 


Program 7.20: Random permutation generator 


Having settled on the test data, we are ready to perform our experl- 
ment. Obtain the worst-case times. From these times you will get a rough 
idea when one function performs better than the other. Now, narrow the 
scope of your experiments and determine the exact value of 2 when one 
sort method outperforms another. For some methods, this value may be 0. 
For instance, each of the other three methods may be faster than Quick Sort 
for all values of n. 2 

Plot your findings on a single sheet of graph paper. Do you see shew 
behavior of Insertion Sort and Quick Sort and the nloga behavior of the 
other two methods for suitably large n (about » > 20)? If not, there is 
something wrong with your test or your clock or with both. For each value 
of n determine the sort function that is fastest (simply look at your graph). 
Write a composite function with the best possible performance for all a. 
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Clock this function and plot the times on the same graph sheet you used 
earlier. 


WHAT TO TURN IN 

You are required to submit a report that states the clock accuracy, the 
number of random permutations tried for Heap Sort, the worst-case data for 
Merge Sort and how you generated it, a table of times for the above values 
of , the times for the narrowed ranges, the graph, and a table of times for 
the composite function. In addition, your report must be accompanied by a 
complete listing of the program used by you (this includes the sorting func- 
tions and the main program for timing and test-data generation). 

Repeat the previous exercise for the case of average runtimes. Average- 
case data are usually very difficult to create, so use random permutations. 
This time, however, do not repeat a permutation many times to overcome 
clock inaccuracies. Instead, use each permutation once and clock the 
overall time (for a fixed n). 


Assume you are given a list of five-letter English words and are faced with 
the problem of listing these words in sequences such that the words in each 
sequence are anagrams (i.e., if x and y are in the same sequence, then word 
x is a permutation of word y). You are required to list out the fewest such 
sequences. With this restriction, show that no word can appear in more 
than one sequence. How would you go about solving this problem? 


Assume you are working in the census department of a small town where 
the number of records, about 3000, is small enough to fit into the intemal 
memory of a computer. All the people currently living in this town were 
born in the United States. There is one record for each person in this town. 
Each record contains (a) the state in which the person was born, (b) county 
of birth, and (c) name of person. How would you produce a list of all per- 
sons living in this town? The list is to be ordered by state. Within each 
State the persons are to be listed by their counties, the counties being 
arranged in alphabetical order. Within each county, the names are also 
listed in alphabetical order. Justify any assumptions you make. 


(Bubble Sort] in a Bubble Sort several left-to-right passes are made over 
the array of records to be sorted. In each pass, pairs of adjacent records are 
compared and exchanged if necessary. The sort terminates following a 
pass in which no records are exchanged. 


(a) Write a C++ function for Bubble Sort. 
(b) What is the worst-case complexity of your function? 
(c) How much time does your function take on a sorted array of records? 


(da) How much time does your function take on an array of records that 
are in the reverse of sorted order? 
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9, Redo the preceding exercise beginning with an unsorted chain of records 
and ending with a sorted chain. 

10. {Programming Project| The objective of this exercise is to study the effect 
of the size of an array element on the computational time of various sorting 
algorithms. 

(a) Use Insertion Sort, Quick Sort, Iterative Merge Sort, and Heap Sort 
to sort arrays of (i) characters (char), (ii) integers (int), (ii) floating 
point numbers (float), and (iv) rectangles (Assume that a rectangle is 
represented by the coordinates of its bottom left point and its height 
and width, all of which are of type float. Assume, also, that rectan- 
gles are to be sorted in non-decreasing order of their areas.) 

{b) Obtain a set of runtimes for each algorithm-data type pair specified 
above. (There should be sixteen such pairs.) To obtain a set of run- 
times of an algorithm-data type pair, you should run the algorithm on 
at least four arrays of different sizes containing elements of the 
appropriate data type. The elements in an array should be generated 
using a random number generator 

(c) Draw tables and graphs displaying your experimental results. What 
do you conclude from the experiments? 


7.10 EXTERNAL SORTING 


7.10.1 Introduction 


In this section, we assume that the lists to be sorted are so large that the whole 
list cannot be contained in the internal memory of a computer, making an inter- 
nal sort impossible. We shall assume that the list (or file) to be sorted resides on 
a disk. The term block refers to the unit of data that is read from or written to a 
disk at one time. A block generally consists of several records. For a disk, there 
are three factors contributing to the read/write time: 


(1) Seek time: time taken to position the read/write heads to the eke 
cylinder. This will depend on the number of cylinders across which the 
heads have to move. ; 

(2) Latency time: time until the right sector of the track is under the read/write 
head. 


(3) Transmission time: time to transmit the block of data to/from the disk. 


The most popular method for sorting on external storage devices i Mee 
Sort. This method consists of two distinct phases. First, segments of the inp: 
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list are sorted using a good internal sort method. These sorted segments, known 
as ruts, are written onto external storage as they are generated. Second, the runs 
generated in phase one are merged together following the merge-tree pattem of 
Figure 7.4, until only one run is left. Because the simple merge function merge 
(Program 7.7) requires only the leading records of the two runs being merged to 
be present in memory at one time, it is possible to merge large runs together. It 
is more difficult to adapt the other internal sort methods considered in this 
chapter to external sorting. 


Example 7.12: A list containing 4500 records is to be sorted using a computer 
with an internal memory capable of sorting at most 750 records. The input list is 
maintained on disk and has a block length of 250 records. We have available 
another disk that may be used as a scratch pad. The input disk is not to be writ- 
ten on. One way to accomplish the sort using the general function outlined 
above is to 

(1) Internally sort three blocks at a time (i.e., 750 records) to obtain six 
runs R, to Rg. A method such as Heap Sort, Merge Sort, or Quick Sort could be 
used. These six runs are written onto the scratch disk (Figure 7.20). 


run 5 mun6 


1-750 751-1500 1501-2250 2251-3000 3001-3750 3751-4500 
3 blocks per run 


Figure 7.20: Blocked runs obtained after internal sorting 


(2) Set aside three blocks of internal memory, each capable of holding 250 
records. Two of these blocks will be used as input buffers and the third as an out- 
put buffer. Merge runs R, and R2. This merge is carried out by first reading one 
block of each of these runs into input buffers. Blocks of runs are merged from 
the input buffers into the output buffer. When the output buffer gets full, it is 
written onto the disk. If an input buffer gets empty, it is refilled with another 
block from the same run. After runs R, and R» are merged, R; and Ry and 
finally Rs and Rg are merged. The result of this pass is three runs, each contain- 
ing q 500 sorted records or six blocks. Two of these runs are now merged using 
the inpuVoutput buffers set up as above to obtain a run of size 3000. Finally, this 
tun is merged with the remaining run of size 1500 to obtain the desired sorted list 
(Figure 7.21). 0 
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tun 1 mun 2 run 3 mun4 run 5 run 6 


Figure 7.21: Merging the six runs 


To analyze the complexity of external sort, we use the following notation: 


1,= maximum seek time 
t= maximum latency time 
tpy= time to read or write one block of 250 records 
tyo= time to input or output one block 
Bly thy + bpp 
t= time to internally sort 750 records 
ni = time to merge n records from input buffers to the output buffer 
We shall assume that each time a block is read from or written onto the 
disk, the maximum seek and latency times are experienced. Although this is not 
true in general, it will simplify the analysis. The computing times for the various 
operations in our 4500-record example are given in Figure 7.22. 
The contribution of seek time can be reduced by writing blocks on the 
same cylinder or on adjacent cylinders. A close look at the final computing time 


indicates that it depends chiefly on the number of passes made over the data. In 
addition to the initial input pass made over the data for the internal sort, the 
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Operation time 

(1) read £8 blocks of input, 36tj0 + Otis 
18t;9, internally sort, 6tj5, 
write 18 blocks, 18% 


(2) merge runs | to 6 in pairs 36t9 + 45001, 

(3) merge two runs of 1500 2Atig + 30001, 
records each, 12 blocks 

(4) merge one min of 3000 36179 + 4500tq 
records with one run of 
1500 records 


total time 132tq + 12,0001, + 6t)5 


Figure 7.22: Computing times for disk sort example 


merging of the runs requires 2-2/3 passes over the data (one pass to merge 6 runs 
of length 750 records, two-thirds of a pass to merge two runs of length 1500, and 
one pass to merge one run of length 3000 and one of length 1500). Since one full 
Pass covers 18 blocks, the input and output time is 2x (2-23 
+ 1) 18 iq = 132tj9. The leading factor of 2 appears because each record that 
is read is also written out again. The merge time is 2-2/3 x 4500ty = 12,000%,. 
Because of this close relationship between the overall computing time and the 
number of passes made over the data, future analysis will be concemed mainly 
with counting the number of passes being made. Another point to note regarding 
the above sort is that no attempt was made to use the computer's ability to carry 
out inpuVoutput and CPU operation in parallel and thus overlap some of the 
time. In the ideal situation we would overlap almost all the input/output time 
with CPU processing so that the real time would be approximately 
132 tig = 12,000 t,, + Ot. 

If we have two disks, we can write on one, read from the other, and merge 
buffer loads already in memory in parallel. A proper choice of buffer lengths and 
buffer handling schemes will result in a time of almost 66t;9. This parallelism is 
an important consideration when sorting is being carried out in a nonmultipro- 
gramming environment. In this situation. unless input/output and CPU process- 
ing is going on in parallel, the CPU is idle during input/output. In a multipro- 
gramming environment, however, the need for the sorting program to carry out 
inpuVoutput and CPU processing in parallel may not be so critical, since the 
CPU can be busy working on another program (if there are other programs in the 
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system at the time) while the sort program waits for the completion of its 
input/output. Indeed, in many multiprogramming environments it may not even 
be possible to achieve parallel input, output, and internal computing because of 
the structure of the operating system. 

The number of merge passes over the runs can be reduced by using a 
higher-order merge than two-way merge. To provide for parallel input, output, 
and merging, we need an appropriate buffer-handling scheme. Further improve- 
ment in runtime can be obtained by generating fewer (or equivalently longer) 
Tuns than are generated by the strategy described above. This can be done using 
a loser tree. The loser-tree strategy to be discussed in Section 7.10.4 results in 
Tuns that are on the average almost twice as long as those obtained by the above 
strategy. However, the generated runs are of varying size. As a result, the order 
in which the runs are merged affects the time required to merge all runs into one. 
We consider these factors now. 


7.10.2 k-Way Merging 


The two-way merge function Merge (Program 7.7) is almost identical to the 
merge function just described (Figure 7.21). In general, if we start with m runs, 
the merge tree corresponding to Figure 7.21 will have [loggm]+1 levels, for a 
total of [logym] passes over the data list. The number of passes over the data 
can be reduced by using a higher-order merge (i.e., k-way merge for k 2 2). In 
this case, we would simultaneously merge k runs together. Figure 7.23 illustrates 
a four-way merge of 16 runs. The number of passes over the data is now two, 
versus four passes in the case of a two-way merge. In general, a k-way merge on 
m runs requires [logy] passes over the data. Thus, the input/output time may 
be reduced by using a higher-order merge. 

The use of a higher-order merge, however, has some other effects on the 
sort. To begin with, k runs of size 54, 52, 53, °° ~, 8, can no longer be merged 


internally in O(})s;) time. In a k-way merge, as in a two-way merge, the next 


i 
record to be output is the one with the smallest key. The smallest has now to be 
found from k possibilities and it could be the leading record in any of the k runs. 
The most direct way to merge k runs is to make k — 1 comparisons t¢ determine 


the next record to output. The computing time for this is O(& — 1) si). Since 


1 ‘ 
logym passes are being made, the total number of key companies 
n(k ~ Ilogym = n(k — I)loggm /logok, where n is the number of records in 
list. Hence, (k ~ !)/logok is the factor by which the number of key Com 
increases. As k increases, the reduction in input/output time will be outweighe: 
by the resulting increase in CPU time needed to perform the k-way merge. 
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Figure 7.23: A four-way merge on 16 runs 


For large k (say, k 26) we can achieve a significant reduction in the 
number of comparisons needed to find the next smallest element by using a loser 
tree with k leaves (see Chapter 5). In this case, the total time needed per level of 
the merge tree is O(n log,k). Since the number of levels in this tree is O(logym), 
the asymptotic internal processing time becomes O(n logok logym) = O{nlogzm). 
This is independent of k. 

In going to a higher-order merge, we save on the amount of input/output 
being carried out. There is no significant loss in internal processing speed. Even 
though the internal processing time is relatively insensitive to the order of the 
merge, the decrease in input/output time is not as much as indicated by the 
reduction to log, passes. This is so because the number of input buffers needed 
to carry out a k-way merge increases with k. Although k +1 buffers are 
sufficient, in the next section we shall see that the use of 2k + 2 buffers is more 
desirable. Since the internal memory available is fixed and independent of k, the 
buffer size must be reduced as & increases. This in turn implies a reduction in the 
block size on disk. With the reduced block size, each pass over the data results 
in a greater number of blocks being written or read. This represents a potential 
increase in inpuVoutput time from the increased contribution of seek and latency 
times involved in reading a block of data. Hence, beyond a certain k value the 
inpuVoutput time will increase despite the decrease in the number of passes 
being made. The optimal value for k depends on disk parameters and the amount 
of internal memory available for buffers. 
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7.10.3 Buffer Handling for Parallel Operation 


If k nuns are being merged together by a k-way merge, then we clearly need at 
least k input buffers and one output buffer to carry out the merge. This, however, 
is not enough if input, output, and internal merging are to be carried out in paral- 
lel. For instance, while the output buffer is being written out, internal merging 
has to be halted, since there is no place to collect the merged records. This can 
be overcome through the use of two output buffers. While one is being written 
out, records are merged into the second. If buffer sizes are chosen correctly, then 
the time to output one buffer will be the same as the CPU time needed to fill the 
second buffer. With only & input buffers, internal merging will have to be held up 
whenever one of these input buffers becomes empty and another block from the 
corresponding run is being read in. This input delay can also be avoided if we 
have 2k input buffers. These 2k input buffers have to be used cleverly to avoid 
reaching a situation in which processing has to be held up because of a lack of 


input records from any one run. Simply assigning two buffers per run does not 
solve the problem. 


Example 7.13: Assume that a two-way merge is carried out using four input 
buffers, in [i], 0 $i < 3, and two output buffers, ou [0] and ou [1]. Each buffer is 
capable of holding two records. The first few records of run 0 have key value 1, 
3,5, 7, 8, 9. The first few records of run 1 have key value 2, 4, 6, 15, 20, 25. 
Buffers in [0} and in [2] are assigned to run 0. The remaining two input buffers 
are assigned to run 1, We start the merge by reading in one buffer load from each 
of the two runs. At this time the buffers have the configuration of Figure 7.24(a). 
Now runs 0 and 1 are merged using records from in [0] and in[1]. In parallel 
with this, the next buffer load from run 0 is input. If we assume that buffer 
lengths have been chosen such that the times to input, output, and generate an 
output buffer are all the same, then when ou [0] is full, we have the situation of 
Figure 7.24(b). Next, we simultaneously output ou [0J, input into in (3} from run 
1, and merge into ou[1]. When ou{1] is full, we have the situation of Figure 
7.24(c). Continuing in this way, we reach the configuration of Figure 7.24(e). 
We now begin to output ov [1], input from run 0 into in [2], and merge into ou [0]. 
During the merge, alt records from run O get used before ow [O} gets full. Merg- 
ing must now be delayed until the inputting of another buffer load from run Ois 
completed. G 


Example 7.13 makes it clear that if 2k input buffers are to suffice, then we 
cannot assign two buffers per run. Instead, the buffer must be floating in the sense 
that an individual buffer may be assigned to any run depending upon need. In the 
buifer assignment strategy we shall describe, there will at any time be at least 
one input buffer containing records from each run. The remaining bufters will be 
filled on a priority basis (i.e., the run for which the k-way merging algorithm will 
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ou[0] ou{t) ou[0] ofl] ou[0} ou[t} 
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in(O) in()) in{0) in(1] in{0) inf] 
7 F 5 z [s] {6 
Z ‘ 7 é 7 15 
in(2) in[3) in) in(B} in] in(3) 
output ou [0] output ou fl} 
merge into ou {0} merge into ou [1] merge into ou [0] 
(a) input into in [2] {b) input into in (3) (c) input into in {0) 
6] iJ : “J iJ 
ou(0]) ou[1) ou[O) oufl) ou[0} ou[l] 
[3s] [- = =| [20] 
9 = 3s - 25 
in{O) in(}) in(O) in(h) in{0)  in(1} 
= =} 7 | . a =i 
i Oa Ge 
in(2] in (3] in(2]) in [3] in[2] in [3] 
output on [0] output ox [1] 
merge into ox [1] merge into ou [0] 
{d) input into in (1) (e) input into in [2] (3) 


Figure 7.24: Example showing that two fixed buffers per run are not enough for 
continued parallel operation 
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Tun out of records first is the one from which the next buffer will be filled). One 
may easily predict which run’s records will be exhausted first by simply compar- 
ing the keys of the last record read from each of the k runs. The smallest such 
key determines this run. We shall assume that in the case of equal keys, the 
Merge process first merges the record from the run with least index. This means 
that if the key of the last record read from mun i is equal to the key of the last 
record read from run j, and i < j, then the records read from i will be exhausted 
before those from j. So, it is possible to have more than two bufferloads from a 
given run and only one partially full buffer from another run. All bufferloads 
from the same run are queued together. Before formally presenting the algorithm 
for buffer utitization, we make the following assumptions about the parallel pro- 
cessing capabilities of the computer system available: 


(1) We have two disk drives and the input/output channel is such that we can 
simultaneously read from one disk and write onto the other. 


(2) While data transmission is taking place between an input/output device and 
a block of memory, the CPU cannot make references to that same block of 
memory. Thus, it is not possible to start filling the front of an output buffer 
while it is being written out. If this were possible, then by coordinating the 
transmission and merging rate, only one output buffer would be needed. 
By the time the first record for the new output block is determined, the first 
record of the previous output block has been written out. 

(3) To simplify the discussion we assume that input and output buffers are of 
the same size. 


Keeping these assumptions in mind, we provide a high-level description of 
the algorithm obtained using the strategy outlined earlier and then illustrate how 
it works through an example. Our algorithm, Program 7.21, merges k-runs, k 2 2, 
using a k-way merge. 2k input buffers and two output buffers are used. Each 
buffer is a continuous block of memory. Input buffers are queued in k queues, 
one queue for each run. It is assumed that each input/output buffer is long 
enough to hold one block of records. Empty buffers are placed on a linked stack. 
The algorithm also assumes that the end of each run has a sentinel record with a 
very large key, say +00. It is assumed that all other records have key value less 
than that of the sentinel record. If block lengths, and hence buffer lengths, are 
chosen such that the time to merge one output buffer load equals the time to read 
a block, then almost all input, output, and computation will be carried out in 
parallel. Jt is also assumed that in the case of equal keys, the k-way merge algo- 
rithm first outputs the record from the run with the smallest index. 


We make the following observations about Program 7.21: 


(1) For large &, determination of the queue that will be exhausted first can be 
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{Steps in buffering algorithm) 
Step 1: Input the first block of each of the & runs, setting up k linked queues, 


each having one block of data. Put the remaining & input blocks into a 
linked stack of free input blocks. Set ou to 0. 


Step 2: Let fastKey (i] be the last key input from run i. Let nextRun be the run 


for which dastKey is minimum. If lastKey [nextRun ] # +0, then initiate 
the input of the next block from run nextRun. 


Step 3: Use a function Kwaymerge to merge records from the k input queues 


into the output buffer ou. Merging continues until either the ourput 
buffer gets full or a record with key +e is merged into ou. If, during this 
merge, an input buffer becomes empty before the output buffer gets full 
or before +eo is merged into ou, the Kwaymerge advances to the next 
buffer on the same queue and retums the empty buffer to the stack of 
empty buffers. However, if an input buffer becomes empty at the same 
time as the output buffer gets full or +0 is merged into ou, the empty 
buffer is left on the queue, and Kwaymerge does not advance to the next 
buffer on the queue. Rather, the merge terminates. 


Step 4: Wait for any ongoing disk inpuVoutput to complete. 
Step 5: If an input buffer has been read, add it to the queue for the appropriate 


tun. Determine the next run to read from by determining NextRun such 
that fastKey [nextRun } is minimum. 


Step 6: If lastKey [nextRun ] # +-, then initiate reading the next block from sun 


nextRun into a free input buffer. 


Step 7: Initiate the writing of output buffer ou. Set ou to 1 — ou. 
Step 8: If a record with key +o has been not been merged into the output buffer, 


go back to Step 3. Otherwise, wait for the ongoing write to complete 
and then terminate. 


Program 7.21: k-way merge with floating buffers 


(2) 
(3) 


found in log.k comparisons by setting up a loser tree for last[i],0Si <k, 
rather than making k — 1 comparisons each time a buffer load is to be read 
in. The change in computing time will not be significant, since this queue 
selection represents only a very small fraction of the total time taken by the 
algorithm. 


For large k, function Kwaymerge uses a tree of losers (see Chapter 5). 
All input and output except for the input of the initial & blocks and the 
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output of the iast block is done concurrently with computing. Since, after k 
runs have been merged, we would probably begin to merge another set of k 
Tuns, the input for the next set can commence during the final merge stages 
of the present set of runs. That is, when lastKey [nextRun ] = +o in Step 6, 
we begin reading one by one the first blocks from each of the next set of k 
Tuns to be merged. So, over the entire sorting of a file the only time that is 
not overlapped with the internal merging time is the time to input the first k 
blocks and that to output the last block. 


(4) The algorithm assumes that all blocks are of the same length. Ensuring this 
may require inserting a few dummy records into the last block of each run 
following the sentinel record with key +22. 


Example 7.14: To illustrate the algorithm of Program 7.21, let us trace through 
it while it performs a three-way merge on the three runs of Figure 7.25. Each run 
consists of four blocks of two records each; the last key in the fourth block of 
each of these three runs is +e°. We have six input buffers and two output buffers. 
Figure 7.26 shows the status of the input buffer queues, the run from which the 
next block is being read, and the output buffer being output at the beginning of 
each iteration of the loop of Steps 3 through 8 of the buffering algorithm. 


Run 0 20 25 | 26 28 (29 30 33 +00 
Runt [23 29] [34 36] [38 60] [70 +=] 


Run2 [24 28] [31 33 [40 43 [50° +e 


Figure 7.25: Three runs 


From line 5 of Figure 7.26 it is evident that during the k-way merge, the test 
for “output buffer full?” should be carried out before the test “input buffer 
emply?”, as the next input buffer for that run may not have been read in yet, so 
there would be no next buffer in that queue. In lines 3 and 4 all six input buffers 
are in use, and the stack of free buffers is empty. 0 


We end our discussion of buffer handling by proving that Program 7.21 is 
correct. 
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Figure 7.26: Buffering example 


Theorem 7.2: The following are true for Program 7.21: 


(1) In Step 6, there is always a buffer available in which to begin reading the 
next block. 


(2) During the k-way merge of Step 3, the next block in the queue has been 
read in by the time it is needed. 
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Proof: (1) Each time we get to Step 6 of the algorithm, there are at most k + 1 
buffer loads in memory, one of these being in an output buffer. For each queue 
there can be at most one buffer that is partially full. If no buffer is available for 
the next read, then the remaining & buffers must be full. This means that all the k 
partially full buffers are empty (as otherwise there will be more than k+1 buffer 
Toads in memory). From the way the merge is set up, only one buffer can be both 
unavailable and empty. This may happen only if the output buffer gets full 
exactly when one input buffer becomes empty. But k > 1 contradicts this. So, 
there is always at least one buffer available when Step 6 is being executed. 

(2) Assume this is false. Let run R; be the one whose queue becomes 
empty during Kwaymerge. We may assume that the last key merged was not +o, 
since otherwise Kwaymerge would terminate the merge rather than get another 
buffer for R;. This means that there are more blocks of records for run R; on the 
input file, and lastKey [i] # +ee, Consequently, up to this time whenever a block 
was output, another was simultaneously read in. Input and output therefore pro- 
ceeded at the same rate, and the number of available blocks of data was always 
k, An additional block is being read in, but it does not get queued until Step 5. 
Since the queue for R; has become empty first, the selection rule for choosing the 
next run to read from ensures that there is at most one block of records for each 
of the remaining k — 1 runs. Furthermore, the output buffer cannot be full at this 
time, as this condition is tested for before the input-buffer-empty condition. 
Thus, fewer than k blocks of data are in memory. This contradicts our earlier 
assertion that there must be exactly & such blocks of data. 0 


7.10.4 Run Generation 


Using conventional internal sorting methods such as those discussed earlier in 
this chapter, it is possible to generate runs that are only as large as the number of 
records that can be held in internal memory at one time. Using a tree of losers, it 
is possible to do better than this. In fact, the algorithm we shall present will, on 
the average, generate runs that are twice as long as obtainable by conventional 
methods. This algorithm was devised by Walters, Painter, and Zalk. In addition 
to being capable of generating longer runs, this algorithm will allow for parallel 
input, output, and internal processing. , 

We assume that inpuVoutput buffers have been set up appropriately for 
maximum overlapping of input, output, and internal processing. Wherever there 
is an inpuVoutput instruction in the run-generation algorithm, it is assumed that 
the operation takes place through the input/output buffers. The run generation 
algorithm uses a tree of losers. We assume that there is enough space to con- 
struct such a tree for k records, r(i], <i <k. Each node, i, in this tree has one 
field i[i]. /[é], | $i <k, represents the loser of the tournament played at node #. 
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Each of the k record positions r[i] has a run number rn{i],0 Si <k. This field 
enables us to determine whether or not r{i) can be output as part of the run 
currently being generated. Whenever the toumament winner is output, a new 
record (if there is one) is input, and the tournament is replayed as discussed in 
Chapter 5. 

Function Runs (Program 7.22) is an implementation of the loser tree stra~ 
egy just discussed. The variables used in this function have the following 
significance: 


r{i],Osi<k .... the k records in the tournament tee 
li], 1<i<k .. loser of the toumament played at node i 
iO]... winner of the tournament 
m{i],Osi<k .. the runoumber to which r[i] belongs 
re... run number of current run 
q .. overall tournament winner 
rq... fun number for r[q] 
rmax_..._ number of runs that will be generated 
lastRec ... last record output 


MAXREC ... arecord with maximum key possible 


The function assumes that the relational operators have been overloaded so that 
when two records of the template type T are compared, the comparison is done 
using their keys. 

The loop of lines 11 to 34 repeatedly plays the tournament outputting 
records, The variable /asrKey is made use of in line 22 to determine whether or 
not the new record input, r(q], can be output as part of the current run. If 
key (q] < lastKey then r{q] cannot be output as part of the current run re, as a 
record with larger key value has already been output in this run, When the tree is 
being readjusted (lines 27 to 33), a record with lower run number wins over one 
with a higher run number. When run numbers are equal, the record with lower 
key value wins. This ensures that records come out of the tree in nondecreasing 
order of their rin numbers. Within the same run, records come out of the tree in 
nondecreasing order of their key values. rmax is used to terminate the function. 
In line 19, when we run out of input, a record with run number rmax + 1 is intro- 
maces When this record is ready for output, the function terminates from line 


Analysis of Runs: When the input list is already sorted, only one run is gen- 
erated. On the average, the run size is almost 2k. The time required to generate 
all the runs for an 7 run Jist is O(n log k), as it takes O(log k) time to adjust the 
loser tree each time a record is output. 0 
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template <class T> 
1 void Runs(T *r) 
2{ 
3 r=new 71k); 
4 int *rn = new int(k], */ = new int{k]; 
5 for (int i= 0; i < k; i++) {/ input records 
6 InputRecord (r[i}); rn[i) = 13 
7 } 
8 InitializeLoserTree (); 
9 Tq=1(0); / tournament winner 
10 int rg =1,rce= 1, rmax = 1; T lastRec = MAXREC; 
11 while(1) (// output runs 
12 if (rq != re) {// end of run 
13 output end of run marker; 
14 if (rq > rmax) return; 
15 else re = rq; 
16 
17 WriteRecord (r (q]); lastRec = r(q); // output record r[q] 
18 — # input new record into tree 
19 if (end of input) rm [g] = rmax + 1; 
20 else { 
24 ReadRecord (r(q}); 
22 if (r [9] < lastRec) // new record belongs to next run 
23 mq) =rmax =rq+1; 
24 else rn(q] = rc; 
25 
26 rg=m(qh 
27 H adjust losers 
28 for (t= (k +q)/25 13  /= 25) # tis initialized to be parent of g 
29 if (rn (/ [81] < rq) Mra (Ee Hh == rq) && (rE <r lg) 
30 {/ ris the winner 
31 swap (q. 1[1]); 
32 rq=rn{q); 
33 } 
34} 
35 delete (} 7; delete [ ] rv ; delete [ } /; 
36} 


Program 7,22: Run generation using a loser tree 
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7.10.5 Optimal Merging of Runs 


The cuns generated by function Runs may not be of the same size. When runs 
are of different size, the ran merging strategy employed so far (i.¢., make com- 
plete passes over the collection of runs) does not yield minimum runtimes. For 
example, suppose we have four runs of length 2, 4, 5, and 15, respectively, Fig- 
ure 7.27 shows two ways to merge these using a series of two-way merges. The 
circular nodes represent a two-way merge using as input the data of the children 
nodes. The square nodes represent the initial runs. We shall refer to the circular 
nodes as internal nodes and the square ones as external nodes. Each figure is a 
merge tree. 


@) ©) 
Figure 7.27: Possible two-way merges 


In the first merge tree, we begin by merging the runs of size 2 and 4 to get 
one of size 6; next this is merged with the run of size 5 to get a run of size 11; 
finally this run of size 11 is merged with the run of size 15 to get the desired 
sorted run of size 26. When merging is done using the first merge tree, some 
records are involved in only one merge, and others are involved in up to three 
merges. In the second merge tree, each record is involved in exactly two merges. 
This corresponds to the strategy in which complete merge passes are repeatedly 
made over the data. 

The number of merges that an individual record is involved in is given by 
the distance of the corresponding external node from the root. So, the records of 
the run with 15 records are involved in one merge when the first merge tree of 
Figure 7.27 is used and in two merges when the second tree is used. Since the 
time for a merge is linear in the number of records being merged, the total merge 
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time is obtained by summing the products of the run lengths and the distance 
from the root of the corresponding external nodes. This sum is called the 


weighted external path length. For the two trees of Figure 7.27, the respective 
weighted external path lengths are 


2-344-345-24+15-1=43 
and 


2-244-24+24+5-2415°2=52 


__ The cost of a k-way merge of n runs of length g;, 1 <i $n, is minimized by 
using a merge tree of degree k that has minimum weighted extemal path length. 
We shall consider the case k = 2 only. The discussion is easily generalized to the 
case k > 2 (see the exercises). 

We briefly describe another application for binary trees with minimum 
weighted external path length. Suppose we wish to obtain an optimal set of 
codes for messages M), -+-, My 4). Each code is a binary string that will be 
used for transmission of the corresponding message. At the receiving end the 
code will be decoded using a decode tree. A decode tree is a binary tree in 
which external nodes represent messages. The binary bits in the code word for a 
message determine the branching needed at each level of the decode tree to 
reach the correct external node. For example, if we interpret a zero as a left 
branch and a one as a right branch, then the decode tree of Figure 7.28 
corresponds to codes 000, 001, 01, and 1 for messages M,, M2, My, and M4, 
respectively. These codes are called Huffman codes. The cost of decoding a 
code word is proportional to the number of bits in the code. This number is 
equal to the distance of the corresponding external node from the root node. If 9; 
is the relative frequency with which message M; will be transmitted, then the 
expected decoding time is 


xX 4d 


Isisn+] 


where d; is the distance of the external node for message M; from the root node. 
The expected decoding time is minimized by choosing code words resulting in a 
decode tree with minimal weighted external path length. . na 

A very nice solution to the problem of finding a binary tree with minimum 
weighted external path length has been given by D. Huffman. This solution 
begins with a min heap of n single-node trees. The single node in each of these 
trees represents one of the provided q,’s and its weight is gj. Huffman’s algo- 
rithm then repeatedly extracts two minimum-weight trees a and b from the min 
heap, combines them into a single binary tree c by creating a new root whose left 
and right subtrees are a and 6 respectively and whose weight is the sum of the 
weights of a and 6, and inserts c into the min-heap. After 1-1 rounds of this 
extract, combine, insert process, the min heap will be left with a single binary 
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Figure 7.28: A decode tree 


tree which is asserted to be a binary tree with minimum weighted extemal path 
length. Program 7.23 gives Huffman’s algorithm as a C++ function. The tem- 
plate class TreeNode in defined in Chapter 5. We use the data field of a node to 
store the weight of the binary tree rooted at that node. It is assumed that the 
function Huffman is a friend of TreeNode as well as of MinHeap. 


template <class T> 
void Huffnan(MinHeap<TreeNode <T>*> heap, int n) 
{// heap is initially a min heap of m single-node binary trees as described above. 
for (int i = 0; i <n-1; i++) 
{// combine two minimum-weight trees 
TreeNode <T > * first = heap . Pop); 
TreeNode <T > *second = heap . Pop(); 
TreeNode <T > *bt = new BinaryTreeNode <T >(first, second, 
first .data + second . data); 
heap . Push (bt); 


} 


Program 7,23: Finding a binary tree with minimum weighted extemal path 
length 


Example 7.15: Suppose we have the weights q; = 2, 92 =3,93=5.q4=7, 
45 = 9, and qe = 13. Then the sequence of trees we would get is given in Figure 
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7.29 (the number in a circular node represents the sum of the weights of external 
nodes in that subtree). 


(d) 


Figure 7.29: Construction of a Huffman tree 


The weighted external path length of this tree is 
2:443-445-3413-247-249-2=93 
By comparison, the best complete binary tree has weighted path length 95. O 


Analysis of Huffinan: The main loop is executed n — 1 times. Each call to Pop 
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and Push requires O(logn) time. Hence, the asymptotic computing time is 
O(nlogn). The correctness proof is left as an exercise. O 


EXERCISES 


1. 


FAL 


(a) 


(b) 


n records are to be sorted on a computer with a memory capacity of 5 
records (S << n)}. Assume that the entire S-record capacity may be 
used for input/output buffers. The input is on disk and consists of m 
runs. Assume that each time a disk access is made, the seek time is 
1, and the latency time is 1. The transmission time is #, per record 
transmitted. What is the total input time for phase two of external 
sorting if a k-way merge is used with internal memory partitioned 
into input/output buffers to permit overlap of input, output, and CPU 
processing as in Buffering (Program 7.21)? 

Let the CPU time needed to merge all the runs together be icpy (we 
may assume it is independent of k and hence constant). Let 
t, = 80 ms, t; = 20ms, n = 200,000, m = 64, t, = 107 sec/record, and 
S = 2000. Obtain a rough plot of the total input time, tana. Versus k. 
Will there always be a value of k for which fcpy * finpur? 

Show that function Huffman (Program 7.23) correctly generates a 
binary tree of minimal weighted external path Jength. 

When n runs are to be merged together using an m-way merge, 
Huffman’s method can be generalized to the following rute: ‘First 
add (1 — 7) mod (m — 1) runs of length zero to the set of runs. Then, 
repeatedly merge the m shortest remaining runs until only one run is 
left.” Show that this rule yields an optimal merge pattern for m-way 
merging. 
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CHAPTER 8 


Hashing 


8.1 INTRODUCTION 


In this chapter, we again consider the ADT dictionary that was introduced in 
Chapter 5 (ADT 5.3). Examples of dictionaries are found in many applications, 
including the spelling checker, the thesaurus, the index for a database, and the 
symbol tables generated by loaders, assemblers, and compilers. When a diction- 
ary with » entries is represented as a binary search tree as in Chapter 5 the dic- 
tionary operations Get, Insert and Delete take O(n) time. These dictionary 
operations may be performed in O(Jog ) time using a balanced binary search 
tee (Chapter 10). In this chapter, we examine a technique, called hashing, that 
enables us to perform the dictionary operations Get, Insert and Delete in OU) 
expected time. We divide our discussion of hashing into two parts: static hash- 
ing and dynamic hashing. 
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8.2 STATIC HASHING 
8.2.1 Hash Tables 


In static hashing the dictionary pairs are stored in a table, At, called the Aash 
table. The hash table is partitioned into b buckets, At{0], ---, Ar[b~ 1]. Bach 
bucket is capable of holding s dictionary pairs (or pointers to this many pairs). 
Thus, a bucket is said to consist of s slots, each slot being large enough to hold 
one dictionary pair. Usually s = 1, and each bucket can hold exactly one pair. 
The address or location of a pair whose key is k is determined by a hash function, 
h, which maps keys into buckets. Thus, for any key k, A(k) is an integer in the 
range 0 through 6 — 1. #(k) is the hash or home address of k. Under ideal condi- 
tions, dictionary pairs are stored in their home buckets. 


Definition: The key density of a hash table is the ratio n/T, where n is the 
number of pairs in the table and T is the total number of possible keys. The /oad- 
ing density or loading factor of a hash table is @=n/sb). 0 


Suppose our keys are at most six characters long, where a character may be 

a decimal digit or an uppercase letter, and that the first character is a leer. Then 

the number of possible keys is T= 26x36! > 1.6 x 10°. Any reasonable 
Osiss 

application, however, uses only a very small fraction of these. So, the key den- 


sity, n/T, is usually very small. Consequently, the number of buckets, b, which 
is usually of the same magnitude as the number of keys, in the hash table is also 
much less than 7. Therefore, the hash function A maps several different keys into 
the same bucket. Two keys, k,, and k, are said to be synonyms with respect to h 
if h(k)) = h(k2). 

__ As indicated earlier, under ideal conditions, dictionary pairs are stored in 
their home buckets. Since many keys typically have the same home bucket, it is 
Possible that the home bucket for a new dictionary pair is full at the time we 
wish to insert this pair into the dictionary. When this situation arises, we say that 
an overflow has occurred. A collision occurs when the home bucket for the new 
pair is not empty at the time of insertion. When each bucket has | slot (i.e. s = 
1), collisions and overflows occur simultaneously. 


Example 8.1: Consider a hash table with b = 26 buckets and s = 2. Assume 
that there are n = 10 distinct keys and that each key begins with a letter. The 
loading factor, a, for this table is 10/52 = 0.19. The hash function 4 must map 
each of the possible keys into one of the numbers 0 to 25. If the internal binary 
representation for the letters A to Z corresponds to the numbers 0 to 25, respec- 
tively, then the function A defined by 4(k) = the first character of & will hash all 
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keys into the hash table. The home buckets for GA, D, A, G, L, A2, Al, A3, A4, 
and E are 6, 3, 0, 6, 11, 0, 0, 0, 0, and 4, respectively. The keys A, Al, A2, A3, 
and A4 are synonyms. So also are G and GA. Figure 8.1 shows the keys GA, D, 


A, G, and A2 entered into the hash table. Though not shown in the figure, L is in 
slot 1 of bucket 11. 


Slot 1 Slot 2 


ANkWN=O 


Figure 8.1: Hash table with 26 buckets and two slots per bucket 


Note that GA and G are in the same bucket and each bucket has two slots. 
Similarly, the synonyms A and A2 are in the same bucket. The next key, Al, 
hashes into bucket 0. This bucket is full and a search of the bucket indicates that 
Al is not in the bucket. An overflow has now occurred. Where in the table 
should Al be entered so that it may be retrieved when needed? 0 


When no overflows occur, the time required to insert, delete or search using 
hashing depends only on the time required to compute the hash function and the 
time to search one bucket. Hence, the insert, delete and search times are 
independent of n, the number of entries in the dictionary. Since the bucket size, 
5, is usually smail (for intemnal-memory tables s is usually 1) the search within a 
bucket is carried out using a sequential search. : 

The hash function of Example 8.1 is not well suited for most practical 
applications because of the very large number of collisions and resulting 
overflows that occur. This is so because it is not unusual to find dictionaries in 
which many of the keys begin with the same letter. Ideally, we would like to 
choose a hash function that is both easy to compute and results in very few 
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collisions. Since the ratio b/T is usually very small, it is impossible to avoid 
collisions altogether. 

In summary, hashing schemes use a hash function to map keys into hash- 
table buckets. It is desirable to use a hash function that is both easy to compute 
and minimizes the number of collisions. Since the size of the key space is usu- 
ally several orders of magnitude larger than the number of buckets and since the 
number of slots in a bucket is smal], overflows necessarily occur. Hence, a 
mechanism to handle overflows is needed. 


8.2.2 Hash Functions 


A hash function maps a key into a bucket in the hash table. As mentioned ear- 
lier, the desired properties of such a function are that it be easy to compute and 
that it minimize the number of collisions. In addition, we would like the hash 
function to be such that it does not result in a biased use of the hash table for ran- 
dom inputs; that is, if k is a key chosen at random from the key space, then we 
want the probability that 4(k) =i to be 1/b for all buckets i. With this stipula- 
tion, a random key has an equal chance of hashing into any of the buckets. A 
hash function satisfying this property is called a uniform hash function. 

Several kinds of uniform hash functions are in use in practice. Some of 
these compute the home bucket by performing arithmetic (e.g., multiplication 
and division) on the key. Since, in many applications, the data type of the key is 
not one for which arithmetic operations are defined (e.g., string), it is necessary 
to first convert the key into an integer (say) and then perform arithmetic on the 
obtained integer. In the following subsections, we describe four popular hash 
functions as well as ways to convert strings into integers. 


8.2.2.1 Division 


This hash function, which is the most widely used hash function in practice, 
assumes the keys are non-negative integers. The home bucket is obtained by 
using the modulo (%) operator. The key k is divided by some number D, and the 
remainder is used as the home bucket for k. More formally, 


Akky=k%D 


This function gives bucket addresses in the range 0 through D — 1, so the hash 
table must have at least 6 = D buckets. Although for most key spaces, every 
choice of D makes A a uniform hash function, the number of overflows on real- 
world dictionaries is criticaly dependent on the choice of D. If D is divisible by 
two, then odd keys are mapped to odd buckets (as the remainder is odd), and 
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even keys are mapped to even buckets. Since real-world dictionaries tend to 
have a bias toward either odd or even keys, the use of an even divisor D results 
in a corresponding bias in the distribution of home buckets. In practice, it has 
been found that for real-world dictionaries, the distribution of home buckets is 
biased whenever D has small prime factors such as 2, 3, 5, 7 and so on. How- 
ever, the degree of bias decreases as the smallest prime factor of D increases. 
Hence, for best performance over a variety of dictionaries, you should select D 
so that it is a prime number. With this selction, the smallest prime factor of D is 
D itself. For most practical dictionaries, a very uniform distribution of keys to 
buckets is seen even when we choose D such that it has no prime factor smaller 
than 20. 

When you write a C++ hash table class for general use, the size of the dic- 
tionary to be accommodated in the hash table is not known. This makes it 
impractical to choose D as suggested above. So, we relax the requirement on D 
even further and require only that D be odd. In addition, we set b equal to the 
divisor D. As the size of the dictionary grows, it will be necessary to increase the 
size of the hash table ht dynamically. To satisfy the relaxed requirement on D, 


array doubling results in increasing the number of buckets (and hence the divisor 
D) from b to 2b + 1. 


8.2.2.2 Mid-Square 


The mid-square hash function determines the home bucket for a key by squaring 
the key and then using an appropriate number of bits from the middle of the 
square to obtain the bucket address; the key is assumed to be an integer. Since 
the middle bits of the square usually depend on all bits of the key, different keys 
are expected to result in different hash addresses with high probability, even 
when some of the digits are the same. The number of bits to be used to obtain 
the bucket address depends on the table size. If r bits are used, the range of 
values is 0 through 2'-1. So the size of hash tables is chosen to be a power of 
two when the mid-square function is used. 


8.2.2.3 Folding 


In this method the key & is partitioned into several parts, ail but possibly the Jast 
being of the same length. These partitions are then added together to obtain the 
hash address for k. There are two ways of carrying out this addition. In the first, 
all but the last partition are shifted so that the least significant bit of each lines up 
with the corresponding bit of the last partition. The different partitions are now 
added together to get h(k). This method is known as shift folding. In the second 
method, folding at the boundaries, the key is folded at the partition boundaries. 
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and digits falling into the same position are added together to obtain A (k). This 
is equivalent to reversing every other partition and then adding. 


Example 8.2: Suppose that k = 12320324111220, and we partition it into parts 
that are three decimal digits long. The partitions are P, = 123, P2 = 203, P3 = 
241, Py = 112, and Ps = 20. Using shift folding, we obtain 


5 
h(k) = ¥,P; = 123 + 203 + 241 + 112 + 20= 699 
1st 
When folding at the boundaries is used, we first reverse P2 and P, to obtain 302 
and 211, respectively. Next, the five partitions are added to obtain A(k) = 123 + 
302 + 241 + 211 +20=897. 0 


8.2.2.4 Digit Analysis 


This method is particularly useful in the case of a static file where all the keys in 
the table are known in advance. Each key is interpreted as a number using some 
radix r. The same radix is used for all the keys in the table. Using this radix, the 
digits of each key are examined. Digits having the most skewed distributions are 
deleted. Enough digits are deleted so that the number of remaining digits is 
small enough to give an address in the range of the hash table. 


8.2.2.5 Converting Keys to Integers 


To use some of the described hash functions, keys need to first be converted to 
nonnegative integers. Since all hash functions hash several keys into the same 
home bucket, it is not necessary for us to convert keys into unique nonnegative 
integers. It is ok for us to convert the strings data, structures, and algorithms into 
the same integer (say, 199). In this section, we consider only the conversion of 
strings into non-negative integers. Similar methods may be used to convert other 
data types into non-negative integers to which the described hash functions may 
be applied. 


Example 8.3: {Converting Strings to Integers] Since it is not necessary to con- 
vert strings into unique nonnegative integers, we can map every string, no matter 
how long, into a 16-bit integer. Program 8.1 shows you one way to do this. 
Program 8.1 converts pairs of characters into a unique integer and then 
sums these unique integers. Although it would have been easier to simply add 
all the characters together (rather than shift every other one by 8 bits), doing so 
would give us integers that are not much more than 8 bits long. For example, 
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unsigned int StringToint(string s) 

{/ Convert s into a nonnegative int that depends on all characters of s. 
int length = (int) s.lengthQ); // number of characters in s 
unsigned int answer = 0; 
if (length % 2 == 1) 

{# length is odd 
answer = s.at(length — 1); 
length~-; 


H Sength is now even 
for (int i = 0; i < length; i += 2) 
{// do two characters at a time 
answer += s.at(i); 
answer += ((int) s.at(i + 1)) << 83 


return answer; 


} 


ee 


Program 8.1; Converting a string into a non-negative integer 


strings that are eight characters long would produce integers up to It bits long. 
Shifting by 8 bits allows us to cover the entire range of integers even with strings 
that are two characters long. O 


The C++ STL provides specializations of the STL template class hash<T> 
that transform instances of type 7 into a nonnegative integer of type size_t. Pro- 
gram 8.2 shows a possible specialization of hash<T> for the case when T is the 


STL class string. This specialization is an alternative to the method of Program 
8.1. 


8.2.3. Secure Hash Functions 


Hash functions with additional properties find several applications in computer 
security. One such application is message authentication. Consider a message M 
that is to be transmitted over an insecure channel from A to B. We want B to be 
confident that the received message is the original message that was transmitted, 
and not a forgery. Assume, for simplicity, that we have a means to tansmut 
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template<> 
class hash<string> { 
public: 
size_t operator()(const string theKey) const 
{// Convert sheKey to a nonnegative integer. 
unsigned long hashValue = 0; 
int length = (int) theKey.length); 
for (int i = 0; i < length; i++) 
hashValue = 5 * hashValue + theKey.at (i); 


return size_t (hashValue); 
af 


Program 8.2: The specialization hash<string> 


Messages much smaller than M securely. For example, we may encrypt the 
smaller message or transmit the smaller message on a more expensive but far 
more secure channel than used for the transmission of a long message. One way 
to accomplish our task is to use a hash function # and transmit M's hash value 
h(M) using the more secure method; the message M is transmitted over the 
insecure channel as before. Suppose message M is altered along the insecure 
channel so that B receives a different message M~ B will now compute h(M‘) 
and compare this with the A(M) value it received. If h(M) and h(M%) are 
different, B wil] know that it did not receive the correct message. Notice that if M 
and M” are different but synonyms, 4(M) = h(M% and B will incorrectly con- 
clude that it has received the correct message. Therefore, in the cited application, 
we want to use a hash function A for which it is difficult for a malicious user who 
has knowledge of M and & to determine a synonym for M. This property is 
called weak collision resistance. 

Other properties for secure hash functions that are useful in other scenarios 
are (1) one-way property: for a given c, it is computationally difficult to find a k 
such that h(k)=c and (2) strong collision resistance: it is computationally 
difficult to find a pair (x,y) such that h(x)=h(y). 

Several cryptographic hash functions with these properties have been 
developed. In addition to the security-related properties mentioned earlier, these 
hash functions have additional features designed to make them practical: (1) A 
can be applied to a block of data of any size, (2) h produces a fixed-length hash 
code, and (3) A (k) is relatively easy to compute for any given k, making both 
hardware and software implementations practical. 
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The Secure Hash Algorithm (SHA) was developed at the National Institute 
of Standards and Technology (NIST) in the USA. We describe the SHA-1 func- 
tion. The input to SHA is any message with maximum length less than 2% bits. 
Its output is a 160-bit code. An outline of the algorithm is shown in Program 8.3. 


Step 1: Preprocess the message so that its length is g*512 bits for some integer 
q. The preprocessing may entail adding a string of zeroes to the end of 
the message. 

Step 2: Initialize the 160-bit output buffer OB, which comprises five 32-bit 
registers A, B, C, D, and E, with A = 67452301, B = efcdab89, C = 
98badcfe, D = 10325476, E = c3d2e1f0 (all values are in hexadecimal). 

Step 3: 

for (int i= 1; i<=q; i++) { 
Let B; = ith block of 512 bits of the message; 
OB = F(OB, B,); 


Step 4: Output OB. 


Program 8.3: SHA algorithm 


The function F in Step 3 itself consists of four rounds of 20 steps each! 
Figure 8.2 shows the form of each of these 80 steps. 


Example 8.4: Assume that the contents of registers A through E are specified as 
above: assume f is (B® C ® D), Kg = 52827999, and Wy = 0000000. Note that 
@® is the exclusive or operator. Let us compute the updated values of the regis- 
ters A through E. 2 

We have B = 67452301 (the original value in A), D = 98badcfe (the original 
value in C), and E = 10325476 (the original value in D). C is obtained by a cir- 
cular left-shift of B by 30 bits (which is equivalent to a circular right-shift by 2 
bits). Since, B= 1110 1111 1100 1101 1010 1011 1000 1001, the updated C is 
O11 1011 1411 OOF] 0110 1010 1110 0010 = 7bf36ae2. ne 

We finally compute the new value in A by adding together five quantities. 
Of these, E, W,, and K, are given, f(B,C,D) = B @ C ® D = 67452301, and 
S°(A) = e8a4602c. Adding (mod 2°”) gives 6e 3e de b6. 0 


Example 8.5: The number of times the atomic SHA operation is executed for a 
1KB message is determined as follows. Since 1KB = 1024 bytes = 16*5 12 bits, 
the preprocessing step, Step 1, doesn’t add bits to the message and q = 16 in Pro- 
gram 8.3. Hence, the for loop of Step 3 of the SHA algorithm is iterated 16 
times. Each iteration consists of 80 atomic SHA ops, so the number of atomic 
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t = step number; 0<7<79 

S(B,C,D) = primitive (bitwise) logical function for step 1; 
e.g.,(BAC)V(B AD) : 

s = circular left shift of the 32-bit register by k bits 

WwW, = a 32-bit value derived from B; 

K, = a constant 


Figure 8.2: Atomic SHA operation 
operations is 16 * 80 = 1280. 0 


8.2.4 Overfiow Handling 


8.2.4.1 Open Addressing 


There are two popular ways to handle overflows: open addressing and chaining. 
In this section, we describe four open addressing methods—linear probing, 
which also is known as linear open addressing, quadratic probing, rehashing and 
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random probing. In linear probing, when inserting a new pair whose key is k, we 
search the hash table buckets in the order, ht {h(k) + i] % b, OS i <b — 1, where 
his the hash function and 6 is the number of buckets. This search terminates 
when we reach the first unfilled bucket and the new pair is inserted into this 
bucket. In case no such bucket is found, the hash table is full and it is necessary 
to increase the table size. In practice, to ensure good performance, table size is 
increased when the loading density exceeds a prespecified threshold such as 0.75 
rather than when the table is full. Notice that when we resize the hash table, we 
must change the hash function as well. For example, when the division hash 
function is used, the divisor equals the number of buckets. This change in the 
hash function potentially changes the home bucket for each key in the hash table. 
So, all dictionary entries need to be remapped into the new larger table. 


Example 8.6: Assume we have a 26-bucket table with one slot per bucket and 
the following keys: GA, D, A, G, L, A2, Al, A3, Ad, Z, ZA, E. For simplicity 
we choose the hash function A (k) = first character of k. Initially, all entries in the 
table are null. Since h(GA) = 6, and ht [6] is empty, GA is entered into Az [6]. D 
and A get entered into the buckets Ar{3] and Ar [0], respectively. The next key G 
has 4(G) = 6. This bucket is already used by GA. The next vacant slot is hr{7], 
so G is entered there. L enters ft{ 11]. A2 collides with A at hr [0], the bucket 
overflows, and A2 is entered at the next vacant slot, At[1]. Al, A3, and A4 are 
entered at ht(2}, Ar(4], and At[5], respectively. Z is entered at Ar[25], ZA at 
hu (8] (the hash table is used circularly), and E collides with A3 at Ar [4] and is 
eventually entered at hr (9]. Figure 8.3 shows the resulting table. O 


When s = I and linear probing is used to handle overflows, a hash table 
search for the pair with key & proceeds as follows: 


(1) Compute & (k). 
(2) Examine the hash table buckets in the order hr (h(k)], he[(a(k) + 1) % 5). 
+++ ht((a(k) + 7) % b] until one of the following happens: 
(a) The bucket he [(A(k) + j) % b] has a pair whose key is k; in this case, 
the desired pair has been found. 
{b) At[h(k) + j) is empty; & is not in the table. 


(c) We return to the starting position Ar[/(k)]; the table is full and k is 
not in the table. 


Program 8.4 is the resulting search function. This function assumes that the hash 
table hr stores pointers to dictionary pairs. 

In Example 8.6 we saw that when linear probing is used to resolve 
overflows, keys tend to cluster together. Moreover, adjacent clusters tend to 
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CAHAKANAN YO 


Figure 8.3: Hash table with linear probing (26 buckets, one slot per bucket) 


coalesce, thus increasing the search time. To locate the key ZA in the table of 
Figure 8.3, it is necessary to examine Ar [25], he[0), ---, At[8] — a total of 10 
comparisons. This is far worse than the worst-case behavior for the tree tables 
we shall see in Chapter 10. If each of the keys in the table of Figure 8.3 is 
retrieved exactly once, then the number of buckets examined is I for A, 2 for A2, 
3 for Al, 1 for D, 5 for A3, 6 for A4, 1 for GA, 2 for G, 10 for ZA, 6 for E, 1 for 
L, and | for Z — for a total of 39 buckets examined. The average number of 
buckets examined in a successful search of our example hash table is 3.25. 

When linear probing is used together with a uniform hash hash function, 
the expected average number of key comparisons to look up a key is approxi- 
mately (2 — «)A2 — 2), where a is the loading density. This is the average over 
all possible sets of keys yielding the given loading density and using a uniform 
function h. In Example 8.6, & = 12/26 = .47 and the average number of key 
comparisons is 1.5. Even though the average number of comparisons is small, 
the worst case can be quite large. 
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template <class K, class E> 
pair <K, E>* LinearProbing <K, E >::Get(const K& k) 
{// Search the linear probing hash table At (each bucket has exactly one slot) for k. 
4 \fa pair with this key is found, return a pointer to this pair; otherwise, return 0. 
int i= A(k); // home bucket 
int j; 
for G 


#5 he Lj] && het (j | first '= ks) { 
J=G+1) %b; # treat the table as circular 
if j =i) return 0; 4 back to start point 


} 
if (ht j |ofirst == &) return Ar {j}; 
return 0; 


Program 8.4: Linear probing 


Some improvement in the growth of clusters and hence in the average 
number of probes needed for searching can be obtained by quadratic probing. 
Linear probing was characterized by searching the buckets (i(k) +i) % b 
O<i <b - 1, where b is the number of buckets in the table. In quadratic probing, 
a quadratic function of i is used as the increment. In particular, the search is car- 
ried out by examining buckets h(k), (A(k) + i?) % b, and (A(k)~ i?) % b for 
1 <i (b-1)/2. When dis a prime number of the form 4j + 3, for j an integer, 
the quadratic search described above examines every bucket in the table. Figure 
8.4 lists some primes of the form 4j + 3. 


{Prime | j [| Prime | j 
3 |o} 4 10 
7 s]} so i 14 
1} 2] 127 | 30 

, 19 | 4] 2st | 62 
23. | 5} 503 | 125 
31 | 7 |] 1019 | 254 


Figure 8.4: Some primes of the form 4j + 3 


An alternative method to retard the growth of clusters is to use a series of 
hash functions ft), ty, ---, My. This method is known as rehashing. Buckets 
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hy(k), 1S i Sm are examined in that order. Yet another alternative, random prob- 
ing, is explored in the exercises. 


8.2.4.2 Chaining 


Linear probing and its variations perform poorly because the search for a key 
involves comparison with keys that have different hash values, In the hash table 
of Figure 8.3, for instance, searching for the key ZA involves comparisons with 
the buckets At[0] to At [7], even though none of the keys in these buckets had a 
collision with Ar[25] and so cannot possibly be ZA. Many of the comparisons 
can be saved if we maintain lists of keys, one list per bucket, each list containing 
all the synonyms for that bucket. If this is done, a search involves computing the 
hash address h(k) and examining only those keys in the list for h{k). Although 
the list for h(k) may be maintained using any data structure that supports the 
search, insert and delete operations (e.g., arrays, chains, search wees), chains are 
most frequently used. We typically use an array Ar[0:b-1] of type 
ChainNode <pair <K, E> >* with ht[i] pointing to the first node of the chain 
for bucket i. Program 8.5 gives the search algorithm for chained hash tables. 


template <class K, class E> 
pair <K, E>* Chaining <K, E >::Get(const K& k) 
{// Search the chained hash table hr for k. 
4 Wf a pair with this key is found, return a pointer to this pair; otherwise, return 0. 
int i=h(k); # home bucket 
4 search the chain he [i] 
for (ChainNode <pair <K, E>>* current = ht[i); current; 
current = current ~link) 
if (current data. first == k) return &current data; 
return 0; 


Program 8.5: Chain search 


When chaining is used on the data of Example 8.6, the hash chains of Fig- 
ure 8.5 are obtained. To insert a new key, &, into a chain, we must first vecify that 
it is not currently on the chain. Following this, k may be inserted at any position 
of the chain. In the example of Figure 8.5, new keys were inserted at the front of 
the chains. Deletion from a chained hash table can be done by removing the 
appropriate node from its chain. 

The number of comparisons needed to search for any of the keys in the 
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Figure 8.5: Hash chains corresponding to Figure 8.3 


chained hash table of Figure 8.5 is one for each of A4, D, E, G, L, and ZA; two 
for cach of A3, GA, and Z; three for A1; four for A2; and five for A, fora total of 
24. The average is now two, which is considerably less than the average for 
linear probing. 

When chaining is used along with a uniform hash function, the expected 
average number of key comparisons for a successful search is = l+ a2, where a 
is the loading density n/b (b = number of buckets). For & = 0.5 this number is 
1.25, and for & = 1 it is 1.5. The corresponding numbers for linear probing are 
1.5 and 5, the table size. : 

The performance results cited in this section tend to imply that provided we 
use a uniform hash function, performance depends only on the method used to 
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handle overflows, Although this is true when the keys are selected at random 
from the key space, it is not true in practice. In practice, there is a tendency to 
make a biased use of keys. Hence, in practice, different hash functions result in 
different performance. Generally, the division hash function coupled with chain- 
ing yields best performance. 

The worst-case number of comparisons needed for a successful search 
remains O(7) regardless of whether we use open addressing or chaining. The 
worst-case number of comparisons may be reduced to O(logn) by storing 
synonyms in a balanced search tree (see Chapter 10) rather than in a chain. 


8.2.5 Theoretical Evaluation of Overfiow Techniques 


The experimental evaluation of hashing techniques indicates a very good perfor- 
mance over conventional techniques such as balanced trees. The worst-case 
performance for hashing can, however, be very bad. In the worst case, an inser- 
tion or a search in a hash table with n keys may take O(n) time. In this section, 
we present a probabilistic analysis for the expected performance of the chaining 
method and state without proof the results of similar analyses for the other 
overflow handling methods. First, we formalize what we mean by expected per- 
formance, 

Let hr[0:b — 1] be a hash table with 5 buckets, each bucket having one 
slot. Let A be a uniform hash function with range [0, 5 - 1). If 1 keys k), ko, 
+++, k, are entered into the hash table, then there are 5” distinct hash sequences 
hk), Alka), +>, h(k,). Assume that each of these is equally likely to occur. 
Let S,, denote the expected number of key comparisons needed to locate a ran- 
domly chosen k;, 1 Sin. Then, S, is the average number of comparisons 
needed to find the jth key k,, averaged over 1 < j Sn, with each j equally likely, 
and averaged over all b” hash sequences, assuming each of these also to be 
equally likely. Let U, be the expected number of key comparisons when a 
search is made for a key not in the hash table. This hash table contains n keys. 
The quantity U,, may be defined in a manner analogous to that used for S,. 


Theorem 8.1: Let « = n/b be the loading density of a hash table using a uni- 
form hashing function 4. Then 


(1) for linear open addressing 
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(2) for rehashing, random probing, and quadratic probing 
U, = 1A1-a) 


1 
=-|—Jh a 
Sy (| jog-(1—a) 
(3) for chaining 


U, =O 
S, = 1+ 0/2 


Proof: Exact derivations of U, and S, are fairly involved and can be found in 
Knuth’s book The Art of Computer Programming: Sorting and Searching (see 
the References and Selected Readings section). Here we present a derivation of 
the approximate formulas for chaining. First, we must make clear our count for 
U, and S,. If the key & being sought has /(k) = i, and chain i has q nodes on it, 
then g comparisons are needed if k is not on the chain. If X is in the jth node of 
the chain, | < j $q, then j comparisons are needed. 

When the 7 keys are distributed uniformly over the b possible chains, the 
expected number in each chain is n/b =a. Since U, equals the expected 
number of keys on a chain, we get U,, = a. 

When the ith key, &;, is being entered into the table, the expected number 
of keys on any chain is (i - 1)/b. Hence, the expected number of comparisons 
needed to search for k; after all m keys have been entered is 1 + (/— 1b (this 
assumes that new entries will be made at the end of the chain). Thus, 


1 a . n-1 coq 
=> +(i- =l+ zi+— 0 
Si nyt G-l}=l 2 2 


EXERCISES 


1. Show that the hash function h(k) = k % 17 does not satisfy the one-way 
property, weak collision resistance, or strong collision resistance. 

2. Consider a hash function #(k)=k % D, where D is not given. We want to 
figure out what value of D is being used. We wish to achieve this using as 
few attempts as possible, where an attempt consists of supplying the func- 
tion with & and observing f(k). Indicate how this may be achieved in the 
following two cases. 

(a) Dis known to be a prime number in the range {10,20}. 
(b) Dis of the form 2*, where k is an integer in [1,5]. 
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Write a C++ function to delete the pair with key & from a hash table that 

uses linear probing. Show that simply setting the slot previously occupied 

by the deleted pair to empty does not solve the problem. How must Get 

(Program 8.4) be modified so that a correct search is made in the situation 

when deletions are permitted? Where can a new key be inserted? 

(a) Show that if quadratic searching is carried out in the sequence 
(atk) +7), (hk) + G- 1), 2, A+ 1), AK), (H@)- 1), 
oak) q) with g = (b — 1)/2, then the address difference % b 
between successive buckets being examined is 


b-2,b-4,b-6,...,5,3,1,1,3,5,....6-6,b-4,b-2 


(6) Write a C++ function to search a hash table Ar of size b for the key k. 
Use fA as the hash function and the quadratic probing scheme dis- 
cussed in the text to resolve overflows. Use the results of part (a) to 
reduce the computations. 

[Morris 1968) In random probing, the search for a key, k, in a hash table 

with 6 buckets is carried out by examining the buckets in the order h(k), 

(h(k) + 5(é)) % b, 1 Si< b—1 where s(i) is a pseudo random number. The 

random number generator must satisfy the property that every number from 

1 to’ — 1 must be generated exactly once as i ranges from | tob—-1. 


(a) Show that for a table of size 2’, the following sequence of computa- 
tions generates numbers with this property: 


Initialize g to 1 each time the search routine is called. 

On successive calls for a random number do the following: 
q*=5 

q = low order r + 2 bits of g 

sti)=q/4 


(b) Write search and insert functions for a hash table using random prob- 
ing and the mid-square hash function. Use the random number gen- 
erator of (a). 


It can be shown that for this method, the expected value for the average 
number of comparisons needed to search for a dictionary pair is 
—(Aa)log(1 — a) for large tables (a is the loading factor). 

Develop a C++ hash table class in which overflows are resolved using a 
binary search tree, Use the division hash function with an odd divisor D 
and array doubling whenever the loading density exceeds a prespecified 
amount. Recall that in this context, array doubling actually increases the 
size of the hash table from its current size b = Dto 2b + I. 
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7, Write a C++ function to list all the keys in a hash table in lexicographic 


order. Assume that linear probing is used. How much time does your func- 
tion take? 


8. Let the binary representation of key k be k,k2. Leti{denote the number 
of | bits in k and let the first bit of k, be t. Let 
lk i= []4|/2] and |x} = {|k|/2]. Consider the following hash function 


h(k) = middle q bits of (k; ®k2) 


where © is the exclusive-or operator. Is this a uniform hash function if 
keys are drawn at random from the space of integers? What can you say 
about the behavior of this hash function in actual dictionary usage? 


9. [T. Gonzalez} Design a dictionary representation that allows you to search, 
insert, and delete in O(1) time. Assume that the keys are integer and in the 
range (0, m) and that m + units of space are available, where n is the 
number of insertions to be made. (Hint: Use two arrays, a(n] and b[m], 
where a [i] will be the (i+1)th pair inserted into the table. If k is the ith 
key inserted, then b{k] =i.) Write C++ functions to search, insert, and 
delete. Note that you cannot initialize the arrays a and b as this would take 
O(n + m) time. 

10. [Z Gonzalez] Let s = {s), 52, «++, 5,} and t= {t1,t2, °*".f,} de two 
sets. Assume 1 $5; $m, 1SiSn, and 1<t,Sm,1SiS<r. Using the idea 
of Exercise 9, write a function to determine if s ¢¢. Your function should 
work in O(r +) time. Since s =r iffs ct and 1 Gs, one can determine in 
linear time whether two sets are the same. How much space is needed by 
your function? 


11. [% Gonzalez] Using the idea of Exercise 9, write an O(n + mm) time fune- 
tion to carry out the task of Verify2 (Program 7.3). How much space does 
your function need? 


12. Using the notation of Section 8.2.4, show that when linear probing is used 
nol 


1 
=—ydY, 
at 
Using this equation and the approximate equality 


n 
b 


Up= 4 [1+ ‘| where @ = 
(l-ay 


show that 
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ae 1 
SF [r+ G-a) 


8.3 DYNAMIC HASHING 
8.3.1 Motivation for Dynamic Hashing 


To ensure good performance, it is necessary to increase the size of a hash table 
whenever its loading density exceeds a prespecified threshold. So, for example, 
if we currently have 6 buckets in our hash table and are using the division hash 
function with divisor D = b, then, when an insert causes the loading density to 
exceed the prespecified threhold, we use array doubling to increase the number 
of buckets to 2b + 1. At the same time, the hash function divisor changes to 
2b + 1. This change in divisor requires us to rebuild the hash table by collecting 
all dictionary pairs in the original smaller size table and reinserting these into the 
new larger table. We cannot simply copy dictionary entries from the smaller 
table into corresponding buckets of the bigger table as the home bucket for each 
entry has potentially changed. For very large dictionaries that must be accessi- 
ble on a 24/7 basis, the required rebuild means that dictionary operations must be 
suspended for unacceptably long periods while the rebuild is in progress. 
Dynamic hashing, which also is known as extendible hashing, aims to reduce the 
rebuild time by ensuring that each rebuild changes the home bucket for the 
entries in only 1 bucket. In other words, although table doubling increases the 
total time for a sequence of n dictionary operations by only O(n), the time 
required to complete an insert that triggers the doubling is excessive in the con- 
text of a large dictionary that is required to respond quickly on a per operation 
basis. The objective of dynamic hashing is to provide acceptable hash table per- 
formance on a per operation basis. 

We consider two forms of dynamic hashing—one uses a directory and the 
other does not—in this section. For both forms, we use a hash function / that 
maps keys into non-negative integers. The range of h is assumed to be 
sufficiently large and we use h(k,p) to denote the integer formed by the p least 
significant bits of A(k). 

For the examples of this section, we use a hash function A(k) that 
wansforms keys into 6-bit non-negative integers. Our example keys will be two 
characters each and / transforms letters such as A, B and C into the bit sequence 
100, 101, and 110, respectively. Digits 0 through 7 are transformed into their 3- 
bit representation. Figure 8.6 shows 8 possible 2 character keys together with 
the binary representation of £(k) for each. For our example hash function, 
A(A0,1) = 0, A(A 1,3) = 1. A(B 1,4) = 1001 = 9, and A(C 1,6) = 110 001 = 49. 
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100 000 
100 001 
101 000 


101 001 
110001 
110010 
110011 
110101 


Figure 8.6: An example hash function 
8.3.2 Dynamic Hashing Using Directories 


We employ a directory, d, of pointers to buckets. The size of the directory 
depends on the number of bits of #(k) used to index into the directory. When 
indexing is done using, say, h(k, 2), the directory size is 2? = 4; when h (k, 5) is 
used, the directory size is 32. The number of bits of 4 (k) used to index the direc- 
tory is called the directory depth. The size of the directory is 2‘, where fis the 
directory depth and the number of buckets is at most equal to the directory size. 
Figure 8.7 (a) shows a dynamic hash table that contains the keys AO, BO, Al, BI, 
C2, and C3. This hash table uses a directory whose depth is 2 and uses buckets 
that have 2 slots each. In Figure 8.7, the directory is shaded while the buckets 
are not. In practice, the bucket size is often chosen to match some physical 
characteristic of the storage media. For example, when the dictionary pairs reside 
on disk, a bucket may correspond to a disk track or sector. 

To search for a key k, we merely examine the bucket pointed to by 
d[h(k,1)], where ris the directory depth. 

Suppose we insert C5 into the hash table of Figure 8.7 (a). Since, 4(C5,2) 
= 01, we follow the pointer, d[01], in position 01 of the directory. This gets us to 
the bucket with Al and B1. This bucket is full and we get a bucket overflow. To 
resolve the overflow, we determine the least u such that &(k,u) is not the same 
for all keys in the overflowed bucket. In case the least u is greater than the direc- 
tory depth, we increase the directory depth to this least u value. This requires us 
to increase the directory size but nol the number of buckets. When the directory 
size doubles, the pointers in the original directory are duplicated so that the 
pointers in each half of the directory are the same. A quadrupling of the direc- 
tory size may be handted as two doublings and so on. For our example, the least 
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(a) depth = 2 (b) depth = 3 (c) depth = 4 


Figure 8.7: Dynamic hash tables with directories 


u for which h{k,u) is not the same for Al, B1, and C5 is 3. So, the directory is 
expanded to have depth 3 and size 8. Following the expansion, d{i] = [i + 4}, 
Osi<4. 

Following the resizing (if any) of the directory, we split the overflowed 
bucket using / (k,u). In our case, the overflowed bucket is split using 4 (k, 3). For 
Al and B1, A(k, 3) = 001 and for C5, A(k, 3) = 101. So, we create a new bucket 
with C5 and place a pointer to this bucket in d[!01]. Figure 8.7 (b) shows the 
result. Notice that each dictionary entry is in the bucket pointed at by the direc- 
tory position # (k, 3), although, in some cases the dictionary entry is also pointed 
at by other buckets. For example, bucket 100 also points to AO and BO, even 
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though h (A 0,3) = h(BO,3) # 000. 

Suppose that instead of C5, we were to insert Cl. The pointer in position 
A(C1,2) = 01 of the directory of Figure 8.7 (a) gets us to the same bucket as 
when we were inserting C5. This bucket overflows. The least « for which /(k,u) 
isn’t the same for Al, B1 and Cl is 4. So, the new directory depth is 4 and its 
new size is 16. The directory size is quadrupled and the pointers d [0:3] are repli- 
cated 3 times to fill the new directory. When the overflowed bucket is split, Al 
and C1 are placed into a bucket that is pointed at by d(0001] and B1 into a 
bucket pointed at by ¢[1001]. 

When the current directory depth is greater than or equal to u, some of the 
other pointers to the split bucket also must be updated to point to the new bucket. 
Specifically, the pointers in positions that agree with the last u bits of the new 
bucket need to be updated. The following example illustrates this. Consider 
inserting A4 (h(A4) = 100 100) into Figure 8.7 (b). Bucket [100] overfiows. 
The least u is 3, which equals the directory depth. So, the size of the directory is 
not changed. Using h(k, 3), AO and BO hash to 000 while A4 hashes to 100. So, 
we create a new bucket for A4 and set d{100] to point to this new bucket. 

As a final insert example, consider inserting Cl into Figure 8.7 (b). 
h(C1,3) = 001. This time, bucket @[001] overflows. The minumum u is 4 and so 
it is necessary to double the directory size and increase the directory depth to 4. 
When the directory is doubled, we replicate the pointers in the first half into the 
second half. Next we split the overflowed bucket using h(k, 4). Since h(k, 4) = 
0001 for Al and Cl and 1001 for B1, we create a new bucket with B] and put 
C1 into the slot previously occupied by B1. A pointer to the new bucket is placed 
in d[1001]. Figure 8.7 (c) shows the resulting configuration. For clarity, several 
of the bucket pointers have been replaced by lowercase letters indicating the 
bucket pointed to. . ; 

Deletion from a dynamic hash table with a directory is similar to insertion. 
Although dynamic hashing employs array doubling, the time for this array dou- 
bling is considerably less than that for the array doubling used in static hashing. 
This is so because, in dynamic hashing, we need to rehash only the entries in the 
bucket that overflows rather than all entries in the table. Further, savings result 
when the directory resides in memory while the buckets are on disk, A search 
requires only I disk access; an insert makes | read and 2 write accesses to the 
disk, the array doubling requires no disk access. 


8.3.3 Directoryless Dynamic Hashing 


As the name suggests, in this method, we dispense with the directory, d, of 
bucket pointers used in the method of Section 8.3.2. Instead, an array, At, of 
buckets is used. We assume that this array is as large as possible and so there is 
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no possibility of increasing its size dynamically. To avoid initializing such a 
large array, we use two variables g and r,0 Sg < 2’, to keep track of the active 
buckets. At any time, only buckets 0 through 2’ + g — | are active, Each active 
bucket is the start of a chain of buckets. The remaining buckets on a chain are 
called overflow buckets. The active buckets 0 through q — | as well as the active 
buckets 2” through 2’ + g — I are indexed using h(k, r + 1) while the remaining 
active buckets are indexed using /(k, r). Each dictionary pair is either in an 
active or an overflow bucket. 

Figure 8.8 (a) shows a directoryless hash table At with r = 2 and g= 9. The 
hash function is that of Figure 8.6, A(B4) = 101 100, and &#(B5) = 10] 101, The 
number of active buckets is 4 (indexed 00, 01, 10, and 11). The index of an 
active bucket identifies its chain. Each active bucket has 2 slots and bucket 00 
contains B4 and AO. There are 4 bucket chains, each chain begins at one of the 4 
active buckets and comprises only that active bucket (i.¢., there are no overflow 
buckets). In Figure 8.8 (a), all keys have been mapped into chains using h (k, 2). 
In Figure 8.8 (b), r = 2 and q = 1; f(k, 3) has been used for chains 000 and 100 
while A(k, 2) has been used for chains 001, 010, and O11. Chain 001 has an 
overflow bucket; the capacity of an overflow bucket may or may not be the same 
as that of an active bucket. 

To search for k, we first compute /(k,r). If i(k,r) < q, then k, if present, is 
in a chain indexed using 4 (k, r + 1). Otherwise, the chain to examine is given by 
a r). Program 8.6 gives the algorithm to search a directoryless dynamic hash 
table. 


if (h(k,r) < q) search the chain that begins at bucket A(k,r +1); 
else search the chain that begins at bucket (kr); 


Program 8.6: Searching a directoryless hash table 


To insert C5 into the table of Figure 8.8 (a), we use the search algorithm of 
Program 8.6 to determine whether or not C5 is in the table already. Chain 01 is 
examined and we verify that C5 is not present. Since the active bucket for the 
searched chain is full, we get an overflow. An overflow is handled by activating 
bucket 2” + q; reallocating the entries in the chain g between g and the newly 
activated bucket (or chain) 2’ + q, and incrementing ¢ by |. In case g now 
becomes 2’, we increment r by } and reset g to 0. The reallocation is done using 
h(k, r+ 1). Finally, the new pair is inserted into the chain where it would be 
searched for by Program 8.6 using the new r and q values. 

For our example, bucket 4 = 100 is activated and the entries in chain 00 (¢ 
= 0) are rehashed using r + | =3 bits. B4 hashes to the new bucket 100 and AO 
to bucket 000. Following this, g = 1 and r = 2. A search for CS would examine 
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chain 1 and so CS is added to this chain using an overflow bucket (see Figure 8.8 
{b)). Notice that at this time, the keys in buckets 001, 010 and O11 are hashed 
using h (k, 2) while those in buckets 000 and 100 are hashed using A (k, 3). 


or overflow ; AO 
OT Ao on) bucket 00 

Al [Al| 

ol BS 001 001 Cl 
10 ie 010) 010 ce 
u|S oul on |? 
100 neguactive 100 Be 
ucket = 

101 BS new active 
C5} bucket 
(a)r=2,g=0 (b) Inset C5,r=2,g=1 (c) Insert Cl, r=2,q=2 


Figure 8,8: Inserting into a directoryless dynamic hash table 


Let us now insert C1 into the table of Figure 8.8 (b). Since, h(C1,2) = 01 = 
q, chain 01 = 1 is examined by our search algorithm (Program 8.6). The search 
verifies that C1 is not in the dictionary. Since the active bucket 01 is full, we get 
an overflow. We activate bucket 2’ + g = 5 = 101 and rehash the keys AI, BS, 
and CS that are in chain g. The rehashing is done using 3 bits. Al is hashed into 
bucket 001 while BS and C5 hash into bucket 101. q is incremented by | and the 
new key C1 is inserted into bucket 001. Figure 8.8 (c) shows the result. 
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EXERCISES 


1. Write an algorithm to insert a dictionary pair into a dynamic hash table that 
uses a directory. 

2. Write an algorithm to delete a dictionary pair from a dynamic hash table 
that uses a directory. 

3. White an algorithm to insert a dictionary pair into a directoryless dynamic 
hash table. 

4, Write an algorithm to delete a dictionary pair from a directoryless dynamic 
hash table. 


8.4 BLOOM FILTERS 
8.4.1 An Application—Differential Files 


Consider an application where we are maintaining an indexed file. For simpli- 
city, assume that there is only one index and hence just a single key. Further 
assume that this is a dense index (i.e., one that has an entry for each record in the 
file) and that updates to the file (inserts, deletes, and changes to an existing 
record) are permitted. It is necessary to keep a backup copy of the index and file 
so that we can recover from accidental loss or failure of the working copy. This 
loss or failure may occur for a variety of reasons, which include corruption of the 
working copy due to a malfunction of the hardware or software. We shall refer 
to the working copies of the index and file as the master index and master file, 
respectively. 

Since updates to the file and index are permitted, the backup copies of 
these generally differ from the working copies at the time of failure. So, it is pos- 
sible to recover from the failure only if, in addition to the backup copies, we 
have a log of all updates made since the backup copies were created. We shall 
call this log the transaction log. To recover from the failure, it is necessary to 
process the backup copies and the transaction log to reproduce an index and file 
that correspond to the working copies at the time of failure. The time needed to 
recover is therefore a function of the sizes of the backup index and file and the 
size of the transaction log. The recovery time can be reduced by making more 
frequent backups. This results in a smaller transaction log. Making sufficiently 
frequent backups of the master index and file is not practical when these are very 
large and when the update rate is very high. 

When only the file (but not the index) is very large, a reduction in the 
recovery time may be obtained by keeping updated records in a separate file 
called the differential file. The master file is unchanged. The master index is, 
however, changed to reflect the position of the most current version of the record 
with a given key. We assume that the addresses for differential-file records and 
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master-file records are different. As a result, by examining the address obtained 
from a search of the master index, we can tell whether the most current version 
of the record we are seeking is in the master file or in the differential file. The 
steps to follow when accessing a record with a given key are given in Program 
8.7(b). Program 8.7(a) gives the steps when a differential file is not used. 

Notice that when a differential file is used, the backup file is an exact 
replica of the master file. Hence, it is necessary to backup only the master index 
and differential file frequently. Since these are relatively small, it is feasible to 
do this. To recover from a failure of the master index or differential file, the tran- 
sactions in the transaction log need to be processed using the backup copies of 
the master file, index, and differential file. The transaction log can be expected 
to be relatively small, as backups are done more frequently. To recover from a 
failure of the master file, we nced merely make a new copy of its backup. When 
the differential file becomes too large, it is necessary to create a new version of 
the master file by merging the old master file and the differential file. This also 
Tesults in a new index and an empty differential file. It is interesting to note that 
using a differential file as suggested does not affect the number of disk accesses 
needed to perform a file operation (see Program 8.7(a,b)). 

Suppose that both the index and the file are very large. In this case the 
differential-file scheme discussed above does not work as well, as it is not feasi- 
ble to backup the master index as frequently as is necessary to keep the transac- 
ion log sufficiently small. We can get around this difficulty by using a 
differential file and a differential index. The master index and master file remain 
unchanged as updates are performed. The differential file contains all newly 
inserted records and the current versions of all changed records. The differential 
index is an index to the differential file. This also has null address entries for 
deleted records. The steps needed to perform a file operation when both a 
differential index and file are used are given in Program 8.7(c). Comparing with 
Program 8.7(a), we see that additional disk accesses are frequently needed, as we 
will often first query the differential index and then the master index. Observe 
that the differential file is much smaller than the master file, so most requests are 
satisfied from the master file. 

When a differential index and file are used, we must backup both of these 
with high frequency. This is possible, as both are relatively small. To recover 
from a loss of the differential index or file, we need to process the transactions in 
the transaction log using the available backup copies. To recover from a loss of 
the master index or master file, a copy of the appropriate backup needs to be 
made. When the differential index and/or file becomes too large, the master 
index and/or file is reorganized so that the differential index and/or file becomes 
empty. 
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Step 1: 
Step 2; 
Step 3: 


Step 1: 
Step 2: 


Step 3: 


Step 1: 
Step 2: 


Step 3: 


Step 1: 


Step 2: 


Step 3: 


Search master index for record address. 
Access record from this master file address. 
If this is an update, then update master index, master file, and 
transaction log. 
(a) No differential file 


Search master index for record address. 
Access record from either the master file or the differential file, 
depending on the address obtained in Step 1. 
If this is an update, then update master index, differential file, and 
transaction log. 

(b) Differential file in use 


Search differential index for record address. If the search is 
unsuccessful, then search the master index. 
Access record from either the master file or the differential file, 
depending on the address obtained in Step 1. 
If this is an update, then update differential index, differential file, and 
transaction log. 

(c) Differential index and file in use 


Query the Bloom filter. If the answer is “‘maybe,”” then search 
differential index for record address. If the answer is ‘‘no’’ or if the 
differential index search is unsuccessful, then search the master 
index. 

Access record from either the master file or the differential file, 
depending on the address obtained in Step 1. 

If this is an update, then update Bloom filter, differential index, 
differential file, and transaction log. 


(d) Differential index and file and Bloom filter in use 


Program 8.7: Access steps 
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8.4.2 Bloom Filter Design 


The performance degradation that results from the use of a differential index can 
be considerably reduced by the use of a Bloom filter. This is a device that 
tesides in internal memory and accepts queries of the following type: Is key & in 
the differential index? If queries of this type can be answered accurately, then 
there will never be a need to search both the differential and master indexes for a 
tecord address, Clearly, the only way to answer queries of this type accurately is 
to have a list of all keys in the differential index. This is not possible for 
differential indexes of reasonable size. 

A Bloom filter does not answer queries of the above type accurately. 
Instead of returning one of “‘yes’’ and ‘‘no’” as its answer, it returns one of 
“maybe” and ‘‘no’’. When the answer is ‘‘no,”’ then we are assured that the key 
k is not in the differential index. In this case, only the master index is to be 
searched, and the number of disk accesses is the same as when a differential 
index is not used. If the answer is ‘‘maybe,” then the differential index is 
searched. The master index needs to be searched only if & is not found in the 
differential index. Program 8.7(d) gives the steps to follow when a Bloom filter 
is used in conjunction with a differential index. . 

A filer error occurs whenever the answer to the Bloom filter query is 
“‘maybe”’ and the key is not in the differential index. Both the differential and 
master indexes are searched only when a filter error occurs. To obtain a perfor- 
mance close to that when a differential index is not in use, we must ensure that 
the probability of a filter error is close to zero. 

Let us take a look at a Bloom filter. Typically, it consists of m bits of 
memory and fh uniform and independent hash functions f,, ***. fy. Each fi 
hashes a key & to an integer in the range [1,m]. Initially al! m filter bits are zero, 
and the differential index and file are empty. When key k is added to the 
differential index, bits f,(k), - > ~, f,(k) of the filter are set to 1. When a query of 
the type ‘Is key k in the differential index?” is made, bits f(k), > * +» fu(k) are 
examined. The query answer is “‘maybe’’ if all these bits are 1, Otherwise, the 
answer is ‘‘no.”” One may verify that whenever the answer is ‘‘no,”’ the key can- 
not be in the differential index and that when the answer is “‘maybe,’’ the key 
may or may not be in the differential index. 

We can compute the probability of a filter error in the following way. 
Assume that initially there are n records and that 1 updates are made. Assume 
that none of these is an insert or a delete. Hence, the number of records remains 
unchanged. Further, assume that the record keys are uniformly distributed over 
the key space and that the probability that an update request is for record é is I/n, 
1 <n. From these assumptions, it follows that the probability that a particular 
update does not modify record i is 1-1/2. So, the probability that none of the w 
updates modifies record i is (1-I/n)“. Hence, the expected number of 
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unmodified records is n (1 —!/2)*, and the probability that the (w +1)’st update is 
for an unmodified record is (1 — I/n)". 

Next, consider bit i of the Bloom filter and the hash function Sp Isjsh. 
Let & be the key corresponding to one of the u updates. Since fj is a uniform 
hash function, the probability that f,(k) # iis 1—1/m. As the h hash functions are 
independent, the probability that f;(k) # i for all h hash functions is (i — Vn)". If 
this is the only update, the probability that bit i of the filter is zero is (1—- In)’. 
From the assumption on update requests, it follows that the probability that bit / 
is zero following the u updates is (1 — 1/m)"". From this, we conclude that if after 
u updates we make a query for an unmodified record, the probability of a filter 
error is (1 -(1~1/m)"")*". The probability, P(u), that an arbitrary query made 
after 4 updates results in a filter error is this quantity times the probability that 
the query is for an unmodified record. Hence, 


P(u) = (1-Imy"(1 - (= Ian 
Using the approximation 


Q-lAy ~e™ 
for large x, we obtain 
P(uy~e™™(1 - ewinyh 

when n and m are large. 

Suppose we wish to design a Bloom filter that minimizes the probability of 
a filter error. This probability is highest just before the master index is reorgan- 
ized and the differential index becomes empty. Let u denote the number of 
updates done up to this time. In most applications, m is determined by the 
amount of memory available, and 7 is fixed. So, the only variable in design is h. 
Differentiating P(u) with respect to h and setting the result to zero yields 


h = (log,2)m/u ~ 0.693m/u 
We may verify that this A yields a minimum for P(u). Actually, since h has to be 


an integer, the number of hash functions to use either is [0.693m/u] or 
[0.693m/u] , depending on which one results in a smaller P(w). 


EXERCISES 
1. By differentiating P(w) with respect to h, show that P(i) is minimized 
when h = (log,2)m/u. 
2. Suppose that you are to design a Bloom filter with minimum P(u) and that 
n= 100,000, m = 5000, and u = 1000. 
(a) Using any of the results obtained in the text, compute the number, h, 
of hash functions to use. Show your computations. 
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{b) Whatis the probability, P(e), of a filter error when A has this value? 
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CHAPTER 9 


Priority Queues 


9.1 SINGLE- AND DOUBLE-ENDED PRIORITY QUEUES 


A priority queue is a collection of elements such that each element has an asso- 
ciated priority. We study two varieties of priority queues—single- and double- 
ended—in this chapter. Single-ended priority queues, which were first studied in 
Section 5.6, may be further categorized as min and max priority queues. As 
noted in Section 5.6.1, the operations supported by a min priority queue are: 


SP1: Return an element with minimum priority. 


SP2: Insert an element with an arbitrary priority. 
SP3: Delete an element with minimum priority. 


The operations supported by a max priority queue are the same as those 
supported by a min priority queue except that in SP] and SP3 we replace 
minimum by maximum. The heap structure of Section 5.6 is a classic data struc- 
ture for the representation of a priority queue. Using a min (max) heap, the 
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minimum (maximum) element can be found in O(1) time and each of the other 
two single-ended priority queue operations may be done in O(log n) time, where 
nis the number of elements in the priority queue. In this chapter, we consider 
several extensions of a single-ended priority queue. The first extension, meldable 
(single-ended) priority queue, augments the operations SP1 through SP3 with a 
meld operation that melds together two priority queues, One application for the 
meld operation is when the server for one priority queue shuts down. At this 
time, it is necessary to meld its priority queue with that of a functioning server. 
Two data structures for meldable priority queues—leftist trees and binomial 
heaps—are developed in this chapter. 

A further extension of meldable priority queues includes operations to 
delete an arbitrary element (given its location in the data structure) and to 
decrease the key/priority (or to increase the key, in case of a max priority queue) 
of an arbitrary element (given its location in the data structure). Two data 
structures—-Fibonacci heaps and pairing heaps—are developed for this exten- 
sion. The section on Fibonacci heaps describes how Fibonacci heaps may be 
used to improve the mun time of Dijkstra’s shortest paths algorithm of Section 
64.1. 

A double-ended priority queue (DEPQ) is a data structure that supports the 
following operations on a collection of elements. 


DP1; Return an element with minimum priority. 
DP2: Return an element with maximum priority. 
DP3: Insert an element with an arbitrary priority. 
DP4: Delete an element with minimum priority. 

DPS: Delete an element with maximum priority. 


So, a DEPQ is a min and a max priority queue rolled into one structure. 


Example 9.1: A DEPQ may be used to implement a network buffer. This buffer 
holds packets that are waiting their turn to be sent out over a network link; each 
packet has an associated priority. When the network link becomes available, a 
packet with the highest priority is transmitted. This corresponds to a DeleteMax 
operation. When a packet arrives at the buffer from elsewhere in the network, it 
is added to this buffer. This corresponds to an Insert operation. However, if the 
buffer is full, we must drop a packet with minimum priority before we can insert 
one. This is achieved using a DelereMin operation. O 


Example 9.2: In Section 7.10, we saw how to adapt merge sort to the extemal 
sorting environment. We now consider a similar adaptation of quick sort, which 
has the best expected run time of all known internal sorting methods. Recall that 
the basic idea in quick sort (Section 7.3) is to partition the elements (0 be sorted 
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into three groups L, M, and R. The middle group M contains a single element 
called the pivot, all elements in the left group Z are < the pivot, and all elements 
in the right group R are 2 the pivot. Following this partitioning, the left and right 
element groups are sorted recursively. 

In an external sort (Section 7.10), we have more elements than can be held 
in the memory of our computer. The elements to be sorted are initially on a disk 
and the sorted sequence is to be left on the disk. When the internal quick sort 
method outlined above is extended to an external quick sort, the middle group 
is made as large us possible through the use of a DEPQ. The extemal quick sort 
Strategy is: 


(1) Read in as many elements as wil fit into an internal DEPQ. The elements 
in the DEPQ will eventually be the middle group of elements. 


(2) Process the remaining elements one at a time. If the next element is < the 
smallest element in the DEPQ, output this next element as part of the left 
group. If the next element is 2 the largest element in the DEPQ, output this 
next element as part of the right group. Otherwise, remove either the max 
or min element from the DEPQ (the choice may be made randomly or alter- 
nately); if the max element is removed, output it as part of the right group; 
otherwise, output the removed element as part of the left group, insert the 
newly input element into the DEPQ. 


(3) Output the elements in the DEPQ, in sorted order, as the middle group. 
(4) Sort the left and right groups recursively. 0 


Program 9.1 defines an abstract base class DEPQ consisting of a pure vir- 
tual member function for each double-ended priority queue operation. There are 
many heap inspired data structures for the representation of a DEPQ. We 
develop just two of these—symmetric min-max heaps and interval heaps—in 
this chapter. 

Throughout this chapter, we assume that priority queue elements are of a 
data type T that supports the relational operators (==, <=, <, etc.) and that these 
operators compare element priorities. So, for example, one element is less than 
another iff it has a smaller priority. 
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template <class T> 

class DEPQ { 

public: 
virtual const 7& GetMin() const = 0; 
virtual const T& GetMax() const = 0; 
virtual void /nsert(const T&) = 0; 
virtual void DeleteMin() = 0; 
virtual void DeleteMax() = 0; 


b 


Program 9.1: Ciass definition of a double-ended priority queue 


9.2 LEFTIST TREES 
9.2.1 Height-Biased Leftist Trees 


Leftist trees provide an efficient implementation of meldable priority queues. 
Consider the meld operation. Let n be the total number of elements in the wo 
Priority queues (throughout this section, we use the term priority queue to mean 
single-ended priority queue) that are to be melded. If heaps are used to represent 
meldable priority queues, then the meld operation takes O(n) time (this may, for 
example, be accomplished using the heap initialization algorithm of Section 7.6). 
Using a leftist tree, the meld operation as well as the insert and delete min (or 
delete max) operations take logarithmic time; the minimum (or maximum) ele- 
ment may be found in O(1) time. 

Leftist trees are defined using the concept of an extended binary tree. An 
extended binary tree is a binary tree in which all empty binary subtrees have 
been replaced by a square node. Figure 9.1 shows two examples of binary trees. 
Their corresponding extended binary trees are shown in Figure 9.2. The square 
nodes in an extended binary tree are called external nodes. The original (circu- 
lar) nodes of the binary tree are called internal nodes. rn 

There are two varieties of leftist trees—height biased (HBLT) and weight 
biased (WBLT). We study HBLTs in this section and WBLTs in the next. HBLTs 
were invented first and are generally referred to simply as leftist trees. We con- 
tinue with this tradition and refer to HBLTs simply as leftist trees in this section. 

Let x be a node in an extended binary tree. Let LeftChild(x) and 
RightChild(x), respectively, denote the left and right children of the internal 
node x. Define shortest (x) to be the length of a shortest path from x to an exter- 
nal node. It is easy to see that shortest (x) satisfies the following recurrence: 


Leftist Trees 493 


(b) 


Figure 9.1: Two binary trees 


Figure 9.2: Extended binary trees corresponding to Figure 9.1 


— | 0 ifx is an external node 
shortest (x) = | 1 + min {shortest (LeftChild (x)), shortest (RightChild(x))} otherwise 


The number outside each internal node x of Figure 9.2 is the value of 
shortest (x). 
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Definition: A leftist tree is a binary tree such that if it is not empty, then 
shortest (LeftChild (x)) 2 shortest (RightChild (x)) 
for every internal node x. 0 


The binary tree of Figure 9.1(a), which corresponds to the extended binary 
tee of Figure 9.2(a), is not a leftist tree, as shortest (LeftChild (C)) = 0, whereas 
Shortest (RightChild(C)) = 1. The binary tree of Figure 9.1(b) is a leftist tree. 


Lemma 9.1: Let r be the root of a leftist tree that has 7 (internal) nodes. 
(a) n> Dshonest(r) _ 


(b) The rightmost root to external node path is the shortest root to external 
node path, Its length is shortest (r) < logo(n+1). 


Proof: (a) From the definition of shortest (r) it follows that there are no extemal 


nodes on the first shortest (r) levels of the leftist tree. Hence, the leftist tree has 
at least 


1 _ gshortest(r) _ 4 


isl 
internal nodes. (b) This follows directly from the definition of a leftist tree. 0 


Leftist trees are represented using nodes that have the following data 
members: leftChild, rightChild, shortest, and data. We assume that the data 
type T of data overloads the relational operators (<, >, ==, etc.) as comparisons 
between element priorities. We note that the concept of an external node is 
introduced merely to arrive at clean definitions. The external nodes are never 
physically present in the representation of a leftist tree. Rather the appropriate 
child data member (leftChild or rightChild) of the parent of an external node is 
set to NULL (or 0). Program 9.2 contains the class definitions of LeftistNode and 
MinLeftistTree. 


Definition: A min leftist rree (max leftist tree) is a leftist tee in which the key 
value in each node is no larger (smaller) than the key values in its children Gf 
any). In other words, a min (max) leftist tree is a leftist tree that is also a min 
(max) wee. 0 


Two min leftist trees are shown in Figure 9.3. The number inside a node x 
is the priority of the element in x, and the number outside x is shortest @). For 
convenience, all priority queue figures in this chapter show only element priority 


Leftist Trees 495 


template <class T> class MinLeftistTree; #/ forward declaration 


template <class 7> 
class LeftistNode { 
friend class MinLeftistTree<T>; 


private: 
T data; 
LeftistNode *leftChild, +rightChild; 
int shortest; 

k 


template <class T> 
lass MinLeftistTree : public MinPQ<T> { 
public: 
4 constructor 
MinLeftistTree(LeftistNode<T> *init = 0) root(init) { }; 


4 the four min leftist tree operations 
const T& GetMin() const; 
void Insert(const T&); 
T& DeleteMin(); 
void Meld(MinLeftistTree<T>*); 
private: 
LeftistNode<T>* Meld(LeftistNode<T>*, LeftistNode<T>*); 
LeftistNode<T> *root; 


% 


Program 9.2: Class definition of a leftist tree 


rather than the complete element. The operations insert, delete min (delete- 
max), and meld can be performed in logarithmic time using a min (max) leftist 
tree. We shall continue our discussion using min leftist wees. 

The insert and delete-min operations can both be performed by using the 
meld operation. To insert an element x into a min leftist tree, we first create a 
min leftist wee that contains the single element x. Then we meld the two min 
leftist trees. To delete the min element from a nonempty min leftist ee, we 
meld the min leftist trees root leftChild and root—rightChild and delete the 
node root. 

The meld operation itself is quite simple. Suppose that two min leftist trees 
are to be melded. First, a new binary tree containing all elements in both trees is 


496 Priority Queues 


(a) 


Figure 9,3; Examples of min leftist trees 


obtained by following the rightmost paths in one or both trees. This binary tree 
has the property that the key in each node is no larger than the keys in its chil- 
dren (if any), Next, the left and right subtrees of nodes are interchanged as 
necessary to convert this binary tree into a leftist tree. 

As an example, consider melding the two min leftist trees of Figure 9.3. To 
obtain a binary tree that contains all the elements in each tree and that satisfies 
the required relationship between parent and child keys, we first compare the 
root keys 2 and 5. Since 2 < 5, the new binary tree should have 2 in its root. We 
shall leave the left subtree of 2 unchanged and meld 2’s right subtree with the 
entire binary tree rooted at 5. The resulting binary tree will become the new 
right subtree of 2. When melding the right subtree of 2 and the binary tree rooted 
at 5, we notice that 5 < 50. So, 5 should be in the root of the melded tree. Now, 
we proceed to meld the subtrees with root 8 and 50. Since 8 < 50 and 8 has no 
right subtree, we can make the subtree with root 50 the right subtree of 8. This 
gives us the binary tree of Figure 9.4(a). Hence, the result of melding the right 
subtree of 2 and the tree rooted at 5 is the tree of Figure 9.4(b). When the tree of 
Figure 9.4(b) is made the sight subtree of 2, we get the binary tree of Figure 
9.A(c). The leftist tree that is made a subtree is represented by shading its nodes, 
in each step. To convert the tree of Figure 9.4(c) into a leftist tree, we begin at 
the last modified root (i.e., 8) and trace back to the overall root, ensuring that 
shortest (LeftChild ()) 2 shortest (RightChild()). This inequality holds at 8 but 
not at 5 and 2. Simply interchanging the left and right subtrees at these nodes 
causes the inequality to hold. The result is the teftist tree of Figure 9.4(d). The 
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pointers that were interchanged are represented by dotted lines in the figure. 


Figure 9.4: Melding the min leftist wees of Figure 9.3 


The function to meld two leftist trees is given in Program 9.3. This func- 
tion makes use of the recursive function Meld (a,b), which melds two nonempty 
leftist trees. This recursive function intertwines the following two steps: 


(1) create a min tree that contains all the elements 
(2) ensure that each node has a left subtree whose shortest value is greater than 
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template <class T> 
void MinLeftistTree<T>::Meld(MinLeftistTree<T> *b) 
{# Combine the min leftist tree b with the given min leftist tree. 
#f bis set to the empty min leftist tree. 
if (root) root = boot; 
else if (b—root) root = Meld(root, b—root); 
broot =0; 


} 


template<class 7> 
LeftistNode<T>* MinLeftistTree<T>::Meld(LeftistNode<T> *a, 
LeftistNode<T> *b) 
{/ Recursive function to meld two nonempty min leftist trees rooted 
Hat a and b. The root of the resulting min leftist tree is returned. 
J set ato be min leftist tree with smaller root. 
if (@—data > b data) swap(a, b); 


/ create binary tree such that the smallest key in each subtree is in the root 
if (!a—rightChild) a->rightChild = b; 
else a-srightChild = Meld(a—rightChild, b); 


# leftist tree property 

if (ta leftChild {| a—vleftChild—sshortest < arightChild—shortest) 
# interchange subtrees 
swap(a-leftChild, a—rightChild); 


Hf set shortest data member 

if (!a-srightChild) ashortest = 1; 

else a shortest = a-rightChild—shortest + 1; 
return a; 


} 


Program 9.3; Melding two min leftist trees 


or equat to that of its right subtree 


Analysis of Meld(<LeftistNode<T>+*, LeftistNode<T>*): Since Meld moves 
down the rightmost paths in the two leftist trees being melded, and since the 
lengths of these paths are at most logarithmic in the number of elements in each 
tree (Lemma 9.1), the melding of two leftist trees with a total of 2 elements is 
done in time O(log 2). 0 
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9.2.2 Weight-Biased Leftist Trees 


‘We arrive at another variety of leftist ee by considering the number of nodes in 
a subtree, rather than the length of a shortest root to extemal node path. Define 
the weight w (x) of node x to be the number of internal nodes in the subtree with 
root x. Notice that if x is an external node, its weight is 0. If x is an intemal 
node, its weight is | more than the sum of the weights of its children. The 
weights of the nodes of the binary trees of Figure 9.2 appear in Figure 9.5. 


| 1 t 
cu} tl Qo oo 
(a) (b) 


Figure 9.5: Extended binary trees of Figure 9.2 with weights shown 


Definition: A binary tee is a weight-biased leftist tree (WBLT) iff at every inter- 
nal node the w value of the left child is greater than or equal to the w value of the 
right child. A max (min) WBLT is a max (min) tree that is also a WBLT. 0 


Note that the binary tree of Figure 9.5(a) is not a WBLT while that of Fig- 
ure 9.5(b) is. 


Lemma 9.2: Let x be any internal node of a weight-biased leftist tree. The 
length, rightmost (x), of the rightmost path from x to an extemal node satisfies 
rightmost (x) $ logy(w (x)+1). 


Proof: The proof is by induction on w(x). When w(x)=1, rightmost (x)=1 and 
log2(w(x)+!)=log22=1. For the induction hypothesis, assume that 
rightmost (x) ¥ logo(w (x)+1) whenever w{x)<a. Let RightChild(x)} denote the 
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right child of x (note that this right child may be an external node). When 
w(x)sn, w (RightChild (x)) $ (n-1)/2 and 


rightmost (x) = \+rightmost (RightChild (x)) 
$ I+log((n-1)/2+1) 
= L+logo(n+1)-1 
= logo(n +1). O 


The insert, delete max, and initialization operations are analogous to the 
corresponding max HBLT operations. However, the meld operation can be done 
in a single top-to-bottom pass (recall that the meld operation of an HBLT per- 
forms a top-to-bottom pass as the recursion unfolds and then a bottom-to-top 
pass in which subtrees are possibly swapped and shortest-values updated). A 
single-pass meld is possible for WBLTs because we can determine the w values 
on the way down and so, on the way down, we can update w-values and swap 
subtrees as necessary. For HBLTs, a node’s new shortest value cannot be deter- 
mined on the way down the tree. 

Experiments indicate that meldable single-ended priority queue operations 
are faster, by a constant factor, when we use WBLTs rather than HBLTs. 


EXERCISES 


1. Give an example of a binary tree that is not a leftist tree. Label the nodes of 
your binary tree with their shortest value. 


2. Let# be an arbitrary binary tree represented using the node structure for a 
leftist tee, 


(a) Write a function to initialize the shortest data member of each node 
int. 

(b) Write a function to convert # into a leftist tree. 

(c) What is the complexity of each of these two functions? 

3. (a) Into an empty min leftist tree, insert elements with priorities 20, 10, 
5, 18, 6, 12, 14, 4, and 22 (in this order). Show the min leftist tree fol- 
lowing each insert. 

(b) Delete the min element from the final min leftist tree of part (a). 
Show the resulting min leftist tree. 

4. Compare the performance of leftist trees and min heaps under the assump- 
tion that the only operations to be performed are insert and delete min. For 
this, do the following: 

{a) Create a random list of n elements and a random sequence of insert 
and delete-min operations of length m. The latter sequence Is 
created such that the probability of an insert or delete-min operation 
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is approximately 0.5. Initialize a min leftist ree and a min heap to 
contain the n elements in the first random list, Now, measure the 
time to perform the m operations using the min leftist tree as welt as 
the min heap. Divide this time by m to get the average time per 
operation. Do this for 2 = 100, 500, 1000, 2000, ---, 5000. Let m 
be 5000. Tabulate your computing times, 

(b) Based on your experiments, make some statements about the relative 
merits of the two priority-queue schemes. 

Write a function to initialize a min leftist tree with n elements. Assume 

that the node structure is the same as that used in the text. Your function 

must run in ©(n) time. Show that this is the case. Can you think of a way 

to do this initialization in Q(n) time such that the resulting min leftist tree 

is also a complete binary tree? 

Futly code and test the class MinLeftistTree of Program 9.2. 


Write a function to delete the element in node x of a min leftist tee. 
Assume that each node has the data members feftChild, rightChild, parent, 
shortest, and data. The parent data member of a node points to its parent 
in the leftist tree. What is the complexity of your function? 

(Lazy deletion] Another way to handle the deletion of arbitrary elements 
from a min leftist tree is to use a bool data member, deleted, in place of the 
parent data member of the previous exercise. When an element is deleted, 
its deleted data member is set to true. However, the node is not physically 
deleted. When a delete-min operation is performed, we first search for the 
minimum element not deleted by performing a limited preorder search. 
This preorder search traverses only the upper part of the tree as needed to 
identify the min element. All deleted elements encountered are physically 
deleted, and their subtrees are melded to obtain the new min leftist tree. 


(a) Write a function to delete the element in node x of a min leftist tree. 


(b) Write another function to delete the min element from a min leftist 
tree from which several elements have been deleted using the former 
function. 


(c) Whatis the complexity of your function of part (b)? Provide this as a 
function of the number of deleted elements encountered and the 
number of elements in the entire tree? 


[Skew heaps] A skew heap is a min tree that supports the min leftist wee 
Operations: insert, delete min, and meld in amortized time (see Section 9.4 
for a definition of amortized time) O(log 1) per operation. As in the case of 
min leftist trees, insertions and deletions are performed using the meld 
operation, which is carried out by following the rightmost paths in the two 
heaps being melded. However, unlike min leftist wees, the left and right 
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subtrees of all nodes (except the last) on the rightmost path in the resulting 
heap are interchanged. 


(a) Write insert, delete min, and meld functions for skewed heaps. 


(b) Compare the running times of these with those for the same opera- 
tions on a min leftist tree. Use random sequences of insert, delete 
min, and meld operations. 


10. [WBLT] Develop the class MinWbit that implements a weight-biased min 
leftist tree. You must include functions to delete and return the min ele- 
ment, insert an arbitrary element, and meld two min WBLTs. Your meld 
function should perform only a top-to-bottom pass over the WBLTs being 
melded. Show that the complexity of these three functions is O(n). Test all 
functions using your own test data. 


11. Give an example of an HBLT that is not a WBLT as wel! as one that is a 
WBLT but not an HBLT. 


93 BINOMIAL HEAPS 


93.1 Cost Amortization 


A binomial heap is a data structure that supports the same functions (i.e., insert, 
delete min (or delete-max), and meld) as those supported by leftist trees. Unlike 
leftist trees, where an individual operation can be performed in O(log ”) time, it 
is possible that certain individual operations performed on a binomial heap may 
take O(n) time. However, if we amortize (spread out) part of the cost of expen- 
sive operations over the inexpensive ones, then the amortized complexity of an 
individual operation is either O(J) or O(log 2) depending on the type of opera- 
tion. 

Let us examine more closely the concept of cost amortization (we shall use 
the terms cost and complexiry interchangeably). Suppose that a sequence 11, 12, 
D1, 13, 14, 15, 16, D2, I7 of insert and delete-min operations is performed. 
Assume that the actual cost of each of the seven inserts is one. By this, we mean 
that each insert takes one unit of time. Further, suppose that the delete-min 
operations D1 and D2 have an actual cost of eight and ten, respectively. So, the 
tota) cost of the sequence of operations is 25. 

In an amortization scheme we charge some of the actual cost of an opera- 
tion to other operations. This reduces the charged cost of some operations and 
increases that of others. The amortized cost of an operation is the total cost 
charged to it. The cost transferring (amortization) scheme is required to be such 
that the sum of the amortized costs of the operations is greater than or equal to 
the sum of their actual costs. If we charge one unit of the cost of a delete-min 
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operation to each of the inserts since the last delete-min operation (if any), then 
two units of the cost of D1 get transferred to I] and 12 (the charged cost of each 
increases by one), and four units of the cost of D2 get transferred to I3 to 16. The 
amortized cost of each of I} to 16 becomes two, that of I7 is equal to its actual 
cost (i.e., one), and that of each of D1 and D2 becomes 6. The sum of the amor- 
tized costs is 25, which is the same as the sum of the actual costs. 

Now suppose we can prove that no matter what sequence of insert and 
delete-min operations is performed, we can charge costs in such a way that the 
amortized cost of each insertion is no more than two and that of each deletion is 
no more than six. This will enable us to make the claim that the actual cost of 
any insert / delete min sequence is no more than 2i + 6d where i and d are, 
respectively, the number of insert and delete min operations in the sequence. 
Suppose that the actual cost of a deletion is no more than ten, and that of an 
insertion is one. Using actual costs, we can conclude that the sequence cost is no 
more than i+ 10d. Combining these two bounds, we obtain 
min(2i + 6d, i + 10d} as a bound on the sequence cost. Hence, using the notion 
of cost amortization, we can obtain tighter bounds on the complexity of a 
sequence of operations. This is useful, because in many applications, we are 
concerned more with the time it takes to perform a sequence of priority queue 
operations than we are with the time it takes to perform an individual operation. 
For example, when we sort using the heap sort method, we are concemed with 
the time it takes to complete the entire sort; not with the time it takes to remove 
the next element from the heap. In applications such as sorting, where we are 
concemed only with the overall time rather than the time per operation, it is ade- 
quate to use a data structure that has a good amortized complexity for each 
Operation type. 

We shall use the notion of cost amortization to show that although indivi- 
dual delete operations on a binomial heap may be expensive. the cost of any 
sequence of binomial heap operations is actually quite small. 


9.3.2 Definition of Binomial Heaps 


As in the case of heaps and leftist trees, there are two varieties of binomial 
heaps, min and max. A min binomial heap is a collection of min trees; a max 
binomial heap is a collection of max tees. We shall explicitly consider min 
binomial heaps only. These will be referred to as B-heaps. Figure 9.6 shows an 
example of a B-heap that is made up of three min trees. 

Using B-heaps, we can perform an insert and a meld in O(1) actual and 
amortized time and a delete min in O(log n) amortized time. B-heaps are 
represented using nodes that have the following data members: degree, child, 
link, and data. The degree of a node is the number of children it has; the child 
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data member is used to point to any one of its children (if any); the link data 
Member is used to maintain singly linked circular lists of siblings. All the chil- 
dren of a node form a singly linked circular list, and the node points to one of 
these children. Additionally, the roots of the min trees that comprise a B-heap 
are linked to form a singly linked circular list. The B-heap is then pointed at by 
a single pointer min to the min tree root with smallest key. Program 9.4 contains 
the class definitions for BinomialNode and BinomialHeap. 

Figure 9.7 shows the representation for the example of Figure 9.6. To 
enhance the readability of this figure, we have used bidirectional arrows to join 
together nodes that are in the same circular list. When such a list contains only 
one node, no such arrows are drawn. Each of the key sets {10}, {6}, {5,4}, {20). 
{15, 30}, {9}, {12, 7. 16}, and {8, 3, 1} denotes the keys in one of the circular 
lists of Figure 9.7. mit is the pointer to the B-heap. Note that an empty B-heap 
has a O pointer. 


Figure 9.6: A B-heap with three min trees 


9.3.3 Insertion into a Binomial Heap 


An element x may be inserted into a B-heap by first putting x into a new node 
and then inserting this node into the circular list pointed at by min. The pointer 
min is reset to this new node only if min is 0 or the key of x is smaller than the 


key in the node pointed at by min. It is evident that these insertion steps can be 
performed in O(1) time. 
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template <class T> class BinomialHeap; !! forward declaration 


template <class T> 
class BinomialNode { 
friend class BinomialHeap<T>; 
private: 
T data; 
BinomialNode<T> *child, *link; 
int degree; 


h 


template <class T> 
class BinomialHeap : public MinPQ<T> { 
public: 
BinomialHeap(BinomialNode<T> *init = 0) min(init) { }; 


the four binomial heap operations 
const T& GerMin() const; 

void Insert(const T&); 

T& DeleteMin(); 

void Meld(BinomialHeap<T>*); 


private: 
BinomialNode<T> *min; 
4 


Program 9.4: Class definition of a binomial heap 


9.3.4 Melding Two Binomial Heaps 


To meld two nonempty B-heaps, we meld the top circular lists of each into a sin- 
gle circular list. The new B-heap pointer is the min pointer of one of the two 
trees, depending on which has the smaller key. This can be determined with a 
single comparison. Since two circular lists can be melded into a single one in 
O(1) time, the total time required to meld two B-heaps is O(1). 
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Figure 9.7; B-heap of Figure 9.6 showing child pointers and sibling lists 


93.5 Deletion of Min Element 


If min is 0, then the B-heap is empty, and a deletion cannot be performed. 
Assume that min is not 0. min points to the node that contains the min element. 
This node is deleted from its circular list. The new B-heap consists of the 
remaining min trees and the sub-min trees of the deleted root. Figure 9.8 shows 
the situation for the example of Figure 9.6. 


® ® 


Figure 9.8: The B-heap of Figure 9.6 following the deletion of the min element 


Before forming the circular list of min tree roots, we repeatedly join 
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together pairs of min trees that have the same degree (the degree of a nonempty 
min tree is the degree of its root). This min tree joining is done by making the 
min tree whose root has a larger key a subtree of the other (ties are broken arbi- 
trarily). When two min trees are joined, the degree of the resulting min wee is 
one larger than the original degree of each min tree, and the number of min trees 
decreases by one. For our example, we may first join either the min trees with 
roots 8 and 7 or those with roots 3 and 12. If the first pair is joined, the min tree 
with root 8 is made a subtree of the min tree with root 7. We now have the min 
tree collection of Figure 9.9, There are three min trees of degree two in this col- 
lection. If the pair with roots 7 and 3 is picked for joining, the resulting min wee 
collection is that of Figure 9.10. Shaded nodes in Figure 9.9 and Figure 9.10 
denote the min tree that was made a subtree in the previous step. Since the min 
trees in this collection have different degrees, the min tree joining process ter- 
minates. 


(7) G 
® © @) >’ 
Ww © 


Figure 9.9: The B-heap of Figure 9.8 following the joining of the two degree- 
one min trees 


The min tree joining step is followed by a step in which the min tree roots 
are linked together to form a circular list and the B-heap pointer is reset to point 
to the min tree root with smallest key. The steps involved in a delete-min opera- 
tion are summarized in Program 9.5. 

Step 1 takes O(1) time. Step 2 can be done in O(1) time by copying over 
the data from the next node (if any) in the circular list and physically deleting 
that node instead. However, since Step 3 requires us to visit all nodes in the cir- 
cular list of roots, it is not necessary to delete min and leave behind a circular 
list. In Step 3, we can simply examine all nodes other than min in the root-level 
circular list. 

Step 3 may be implemented by using an array, tree, that is indexed from 0 
to the maximum possible degree, maxDegree, of a min tree. Initially all entries 
in this array are 0. Let s be the number of min trees in min and y. The lists min 
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Figure 9.10: The B-heap of Figure 9.9 following the joining of two degree-two 
min trees 


Step 1: (Handle empty B-heap] if (min) throw QueueEmpty(; 


Step 2: [Deletion from nonempty B-heap} x = min-~>data; y = min —child, 
delete min from its circular list; following this deletion, min points to 
any remaining node in the resulting list; if there is no such node, then 
min =0; 

Step 3: [Min-tree joining) Consider the min trees in the lists min and y; join 
together pairs of min trees of the same degree until all remaining min 
trees have different degrees; 

Step 4: {Form min tree root list] Link the roots of the remaining min trees (if 
any) together to form a circular list; set min to point to the root (if any) 
with minimum key, return -x; 


Program 9.5: Steps in a delete-min operation 


and y created in Step 2 are scanned. For each min tree p in the lists min and y 
created in Step 2, the code of Program 9.6 is executed. The function JoinMin- 
Trees makes the input tree with larger root a subtree of the other tree. The result- 
ing tree is returned in the first parameter. In the end, the array tree contains 
Pointers to the min trees that are to be linked together in Step 4. Since each time 
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a pair of min tees is joined the total number of min trees decreases by one, the 
number of joins is at most s—1. Hence, the complexity of Step 3 is 
O(maxDegree + 5). 


for (d = p degree; tree[d); d++) 


JoinMinTrees (p, tree (d]); 
tree [d] = 0; 


} 
tree{d) =p; 


Program 9.6: Code to handle min tree p encountered during a scan of lists min 
and y 


Step 4 is accomplished by scanning tree[0],..., tree {maxDegree] and 
linking together the min trees found. During this scan, the min tree with 
minimum key may also be determined. The complexity of Step 4 is O(mawe- 
gree). 


9.3.6 Analysis 


Definition: The binomial tree, By, of degree k is a tree such that if k = 0, then the 
tree has exactly one node, and if k > 0, then the tree consists of a root whose 
degree is k and whose subtrees are Bg, By, --+, By. G 


The min trees of Figure 9.6 are By, By, and B3, respectively. One may ver- 
ify that B, has exactly 2* nodes. Further, if we start with a collection of empty 
B-heaps and perform inserts, melds, and delete mins only, then the min trees in 
each B-heap are binomial trees. These observations enable us to prove that 
when only inserts, melds, and delete mins are performed, we can amortize costs 
such that the amortized cost of each insert and meld is O(1), and that of each 
delete min is O(log 7). 


Lemma 9.3: Let @ be a B-heap with n elements that results from a sequence of 
insert, meld, and delete-min operations performed on a collection of initially 
empty B-heaps. Each min tee in a has degree < logon. Consequently, maxDe- 
gree S |logan], and the actual cost of a delete-min operation is O(log n +5). 


Proof: Since each of the min trees in a is a binomial tree with at most nodes, 
None can have degree greater than {log;n]. O 
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Theorem 9.1: If a sequence of n insert, meld, and delete-min operations is per- 
formed on initially empty B-heaps, then we can amortize costs such that the 
amortized time complexity of each insert and meld is O(1), and that of each 
delete-min operation is O(log #). 


Proof: For each B-heap, define the quantities #insert and LastSize in the follow- 
ing way: When an initially empty B-heap is created or when a delete-min opera- 
tion is performed on a B-heap, its #insert value is set to zero. Each time an insert 
is done on a B-heap, its #insert value is increased by one. When two B-heaps 
are melded, the #insert value of the resulting B-heap is the sum of the #insert 
values of the B-heaps melded. Hence, #insert counts the number of inserts per- 
formed on a B-heap or its constituent B-heaps since the last delete-min operation 
performed in each. When an initially empty B-heap is created, its LastSize value 
is zero. When a delete-min operation is performed on a B-heap, its LastSize is 
set to the number of min trees it contains following this delete. When two B- 
heaps are melded, the LastSize value for the resulting B-heap is the sum of the 
LastSize values in the two B-heaps that were melded. One may verify that the 
number of min trees in a B-heap is always equal to #insert + LastSize. 

Consider any individual delete-min operation in the operation sequence. 
Assume this is from the B-heap a. Observe that the total number of elements in 
all the B-heaps is at most 7, as only inserts add elements, and at most » inserts 
can be present in a sequence of n operations. Let u =a. min ~>degree < logzn. 

From Lemma 9.3, the actual cost of this delete-min operation is 
O(logn +s). The logn term is due to maxDegree and represents the time 
needed to initialize the array tree and to complete Step 4. The s term represents 
the time to scan the lists min and y and to perform the s—1 (at most) min tree 
joins. We see that s = #insert + LastSize + u — 1. If we charge #insert units of 
Cost to the insert operations that contribute to the count #insert and LastSize 
units to the delete mins that contribute to the count LastSize (each such detete- 
min operation is charged a number of cost units equal to the number of min trees 
it left behind), then only u — 1 of the s cost units remain. Since u < logan, and 
since the number of min trees in a B-heap immediately following a delete-min 
operation is $ logan, the amortized cost of a delete-min operation becomes 
O(logz). 

Since this charging scheme adds at most one unit to the cost of any insert, 
the amortized cost of an insert becomes O(1). The amortization scheme used 
does not charge anything extra to. a meld. So, the actual and amortized costs of a 
meld are also O(1). O 


From the preceding theorem and the definition of cost amortization, it fol- 
lows that the actual cost of any sequence of i inserts, ¢ melds, and dm detete-min 
operations js Ofi + ¢ + dm log i). 
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EXERCISES 


1. 


Let S be an initially empty stack. We wish to perform two operations on S: 
Add (x) and DeleteUntil (x). These are defined as follows: 


(a) Add (x): Add the element x to the top of the stack S. This operation 
takes O(1) time per invocation. 

(b) DeleteUntil (x): Delete elements from the top of the stack up to and 
including the first x encountered. If p elements are deleted, the time 
taken is O(p). 


Consider any sequence of n stack operations (Adds and DeleteUntils). 

Show how to amortize the cost of the Add and DeleteUntil operations so 

that the amortized cost of each is O(1). From this, conclude that the time 

needed to perform any such sequence of operations is O(n). 

Let x be an unsorted array of n elements. The function Search (x,n,i,y) 

searches x for y by examining x [i], x [i +1], and so on, in that order, for the 

least j such that x[j] = y. If no such j is found, j is set ton + 1. On termi- 
nation, function Search sets i to j. Assume that the time required to exam- 

ine a single element of x is O(1). 

(a) What is the worst-case complexity of Search? 

(b) Suppose that a sequence of m searches is performed beginning with i 
= 1. Use a cost amortization scheme that assigns costs both to ele- 
ments and to search operations. Show that it is always possible to 
amortize costs so that the amortized cost of each element is O(1) and 
that of each search is also O(1). From this, conclude that the cost of 
the sequence of m searches is O{m +n). 

(a) Into an empty B-heap, insert elements with priorities 20, 10, 5, 18, 6, 
12, 14, 4, and 22 (in this order). Show the final B-heap. 

(b) Delete the min element from the final B-heap of part (a). Show the 
resulting B-heap. Show how you arrived at this final B-heap. 


Fully code and test the class BHeap, which impements a min binomial 
heap. Your class must include the functions GetMin, Insert, DeleteMin, 
and Meld. 


Prove that the binomial tree B, has 2* nodes, k 20. 

Compare the performance of leftist trees and B-heaps under the assumption 

that the only permissible operations are insert and delete min. For this, do 

the following: 

(a) Create a random list of n elements and a random sequence of insert 
and delete-min operations of length m. The number of delete mins 
and inserts should be approximately equal. Initialize a min leftist 
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tree and a B-heap to contain the n elements in the first random list. 
Now, measure the time to perform the m operations using the min 
leftist tree as well as the B-heap. Divide this time by m to get the 
average time per operation. Do this for n = 100, 500, 1000, 2000, 
+++, 5000. Let m be 5000. Tabulate your computing times. 


{b) Based on your experiments, make some statements about the relative 
Terits of the two data structures, 


7. Is the height of every tree in a Binomial heap that has 7 elements O(log 7)? 
If not, what is the worst-case height as a function of n? 


9.4 FIBONACCI HEAPS 
9.4.1 Definition 


There are two varieties of Fibonacci heaps: min and max. A min Fibonacci heap 
is a collection of min trees; a max Fibonacci heap is a collection of max trees. 
We shall explicitly consider min Fibonacci heaps only. These will be referred to 
as F-heaps. B-heaps are a special case of F-heaps. Thus, all the examples of B- 
heaps in the preceding section are also examples of F-heaps. As a consequence, 
in this section, we shall refer to these examples as F-heaps. 

An F-heap is a data structure that supports the four B-heap operations 
GetMin, Insert, DeleteMin, and Meld—as well as the following additional opera- 
tions: 


(1) Delete: Delete the element in a specified node. We refer to this delete 
operation as arbitrary delete. 


(2) DecreaseKey: Decrease the key/priority of a specified node by a given 
positive amount. 


When an F-heap, is used, the Delete operation takes O(log n) amortized 
time and the DecreaseKey takes O(1) amortized time. The B-heap operations 
can be performed in the same asymptotic times using an F-heap as they can be 
using a B-heap. 

To represent an F-heap, the B-heap representation is augmented by adding 
two data members, parent and childCut, to each node. The parent data member 
is used to point to the node's parent (if any). The significance of the childCut 
data member will be described later. In addition, the singly linked circular lists 
are replaced by doubly linked circular lists. This requires us to replace the data 
member link by the data members /eftLink and rightLink. 

In an F-heap, the B-heap operations GetMin, Insert, DeleteMin, and Meld 
are performed exactly as for the case of B-heaps. So, we need consider only the 
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remaining two operations: Delete and DecreaseKey. 


9.4.2 Deletion from an F-Heap 


To delete an arbitrary node b from a F-heap, we do the following: 


(1) If min = b, then do a delete min; otherwise do Steps 2, 3, and 4 below. 
(2) Delete 5 from its doubly linked list. 


(3) Combine the doubly linked list of b's children with the doubly linked list 
pointed at by min into a single doubly linked list. Trees of equal degree are 
not joined as in a delete-min operation. 


(4) Dispose of node b. 
For example, if we delete the node containing 12 from the F-heap of Figure 
9.6, we get the F-heap of Figure 9.11. The actual cost of an arbitrary delete is 


Q(1) unless the min element is being deleted. In this case the deletion time is the 
time for a delete-min operation. 


PAAT 


Figure 9.11; F-heap of Figure 9.6 following the deletion of 12 


9.4.3 Decrease Key 


To decrease the key in node 6 we do the following: 


(i) Reduce the key in b. 
(2) If bis not a min tree root and its key is smaller than that in its parent, then 
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delete from its doubly linked list and insert it into the doubly linked list 
of min tree roots. 


(3) Change min to point to b if the key in b is smaller than that in min. 
Suppose we decrease the key 15 in the F-heap of Figure 9.6 by 4. The 


fesulting F-heap is shown in Figure 9.12. The cost of performing a decrease-key 
operation is O(1). 


Figure 9.12: P-heap of Figure 9.6 following the reduction of 15 by 4 


9.4.4 Cascading Cut 


With the addition of the delete and decrease-key operations, the min trees in an 
F-heap need not be binomial trees. In fact, it is possible to have degree-k min 
trees with as few as k+I nodes. As a result, the analysis of Theorem 9.1 is no 
longer valid. The analysis of Theorem 9.1 requires that each min tree of degree k 
have an exponential (in k) number of nodes. When decrease-key and delete 
Operations are performed as described above, this is no longer true. To ensure 
that each min tree of degree k has at least c* nodes for some c, c > 1, each delete 
and decrease-key operation must be followed by a cascading-cut step. For this, 
we add the bool data member childCur to each node. The value of this data 
member is useful only for nodes that are not the root of a min tree. In this case, 
the childCur data member of node x has the value true iff one of the children of x 
was cut off (i.e., removed) after the most recent time x was made the child of its 
current parent. This means that each time two min trees are joined in a delete- 
min operation, the childCut data member of the root with larger key should be 
set to false. Further, whenever a delete or decrease-key operation deletes a node 
q that is not a min tree root from its doubly linked list (Step 2 of delete and 
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decrease key), then the cascading-cut step is invoked. During this step, we 
examine the nodes on the path from the parent p of the deleted node g up to the 
nearest ancestor of the deleted node with chifdCut = false. If there is no such 
ancestor, then the path goes from p to the root of the min tree containing p. All 
nonroot nodes on this path with childCut data member true are deleted from 
their respective doubly linked lists and added to the doubly linked list of min tree 
root nodes of the F-heap. If the path has a node with childCut data member 
false, this data member is changed to true. 

Figure 9.13 gives an example of a cascading cut. Figure 9.3(a) is the min 
tree containing 14 before a decrease-key operation that reduces this key by 4. 
The childCut data members are shown only for the nodes on the path from the 
parent of 14 to its nearest ancestor with childCut = false. Nodes with chifdCut = 
true are shaded in the figure. Alt unshaded nodes have childCur = false. During 
the decrease-key operation, the min tree with root 14 is deleted from the min tree 
of Figure 9.13(a) and becomes a min tree of the F-heap. Its root now has key 10. 
This is the first min tree of Figure 9.13(b). During the cascading cul, the min 
trees with roots 12, 10, 8, and 6 are cut off from the min tree with root 2. Thus, 
the single min tree of Figure 9.13(a) becomes six min trees of the resulting F- 
heap. The childCur value of 4 becomes true. All other childCut values are 
unchanged. 


9.4.5 Analysis 


Lemma 9.4; Let a be an F-heap with n elements that results from a sequence of 

insert, meld, delete min, delete, and decrease-key operations performed on ini- 

tially empty F-heaps. 

(a) Let b be any node in any of the min trees of a. The degree of b is at most 
loggnt, where = (1+¥5)/2, and mt is the number of elements in the subtree 
with root b. 


(b) maxDegree < |logyn|, and the actual cost of a delete-min operation is 
O(log n + 5). 


Proof: We shail prove (a) by induction on the degree of b. Let N, be the 
minimum number of elements in the subtree with root b when b has degree i. We 
see that No = | and N, = 2. So, the inequality of (a) holds for degrees 0 and 1. 
For i> 1, letc,, ---, ¢; be the i children of 6. Assume that ¢, was made a child 
of b before cy.) 7 <i. Hence, when c¢;, k <i, was made a child of b, the degree 
of b was at least k — 1. The only F-heap operation that makes one node a child of 
another is delete min. Here, during a join min tree step, one min tree is made a 
subtree of another min tree of equal degree. Hence, at the time of joining, the 


516 Priority Queues 


eS 


(a) (b) 


Figure 9.13: A cascading cut following a decrease of key 14 by 4 


degree of c, must have been equal to that of b, Subsequent to joining, its degree 
can decrease as a result of a delete or decrease-key operation. However, follow- 
ing such a join, the degree of cy can decrease by at most one, as an attempt to cut 
off a second child of cy results in a cascading cut at cx. Such a cut causes cx to 
become the root of a min tree of the F-heap. Hence, the degree, dj, of cy is at 
least max{0,&—2}. So, the number of elements in cy is at least Ny,. This 
implies that 

id i2 

Nj=No+ DN + l= DM +2 
£20 k=O 
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One may show (see the exercises) that the Fibonacci numbers satisfy the equality 
Aw? 
Fy, = DF_t+i.h > 1, Fo =0,and F, =1 
k=O 
From this we may obtain the equality N; = F;,2,i20. Further, since F,42 2’, 
N,26!. Hence, iS loggm. (b) is a direct consequence of (a). 0 


Theorem 9.2: If a sequence of n insert, meld, delete min, delete, and decrease- 
key operations is performed on an initially empty F-heap, then we can amortize 
costs such that the amortized time complexity of each insert, meld, and 
decrease-key operation is O(1) and that of each delete min and delete operation 
is O(log n). The total time complexity of the entire sequence is the sum of the 
amortized complexities of the individual operations in the sequence. 


Proof: The proof is similar to that of Theorem 9.1. The definition of #insert is 
unchanged. However, that of LastSize is augmented by requiring that following 
each delete and decrease-key operation, LastSize be changed by the net change 
in the number of min trees in the F-heap (in the example of Figure 9.13 LastSize 
is increased by 5). With this modification, we see that at the time of a delete-min 
Operation s = #insert + LastSize + u—1. #insert units of cost may be charged, 
one each, to the #insert insert operations that contribute to this count, and Last- 
Size units may be charged to the delete min, delete, and decrease-key operations 
that contribute to this count. This results in an additional charge of at most loggn 
to each contributing delete min and delete operation and of one to each contri- 
buting decrease-key operation. As a result, the amortized cost of a delete-min 
operation is O(log 7). 

Since the total number of cascading cuts is limited by the total number of 
deletes and decrease-key operations (as these are the only operations that can set 
childCut to true), the cost of these cuts may be amortized over the delete and 
decrease-key operations by adding one to their amortized costs. The amortized 
cost of deleting an element other than the min element becomes O(log 1), as its 
actual cost is O(1) (excluding the cost of the cascading-cut sequence that may be 
performed); at most one unit is charged to it from the amortization of all the cas- 
cading cuts; and at most logy units are charged to it from a delete-min opera- 
ion. 

The amortized cost of a decrease-key operation is O(1), as its actual cost is 
O(t) (excluding the cost of the ensuing cascading cut}; at most one unit is 
charged to it from the amortization of all cascading cuts; and at most one unit is 
charged from a delete-min operation. 

The amortized cost of an insert is O(1), as its actual cost is one, and at most 
one cost unit is charged to it from a delete-min operation. Since the amortization 
scheme transfers no charge to a meld operation, its actual and amortized costs 
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are the same. This cost is O(1). Q 


From the preceding theorem, it follows that the complexity of any sequence 
of F-heap operations is O(i + ¢ + dk + (dm +d) log i) where i, c, dk, din, and d 
are, respectively, the number of insert, meld, decrease-key, delete min, and 
delete operations in the sequence. 


9.4.6 Application to the Shortest-Paths Problem 


We conclude this section on F-heaps by considering their application to the 
single-source/all-destinations algorithm of Chapter 6. Let S be the set of vertices 
to which a shortest path has been found and let dist (i) be the length of a shortest 
path from the source vertex to vertex i, i S, that goes through only vertices in A 
On each iteration of the shortest-path algorithm, we need to determine an i, i€ S, 
such that dist (é) is minimum and add this i to S. This corresponds to a delete min 
operation on S. Further, the dist values of the remaining vertices in S may 
decrease. This corresponds to a decrease-key operation on each of the affected 
vertices. The total number of decrease-key operations is bounded by the number 
of edges in the graph, and the number of delete-min operations is n-2. S 
begins with n — | vertices. If we implement S as an F-heap using dist as the key, 
then n ~ | inserts are needed to initialize the F-heap. Additionally, m — 2 delete- 
min operations and at most ¢ decrease-key operations are needed. The total time 
for all these operations is the sum of the amortized costs for each. This is 
O(n log n +e). The remainder of the algorithm takes O(n) time. Hence if an F- 
heap is used to represent S, the complexity of the shortest-path algorithm 
becomes O(n log n +e). This is an asymptotic improvement over the implemen- 
tation discussed in Chapter 6 if the graph does not have Q(7) edges. If this 
single-source algorithm is used n times, once with each of the vertices in the 
graph as the source, then we can find a shortest path between every pair of ver- 
tices in O(n? logn +ne) time. Once again, this represents an asymptotic 
improvement over the O(n>) dynamic programming algorithm of Chapter 6 for 
graphs that do not have 2(n7) edges. It is interesting to note that O(# log n + e) 
is the best possible implementation of the single-source algorithm of Chapter 6, 
as the algorithm must examine each edge and may be used to sort » numbers 
(which takes O(n log 2) time). 


EXERCISES 


1, Fully code and test the class FHeap, which impements a min Fibonacci 
heap. Your class must include the functions GerMin, Insert, DeleteMin, 
Meld, Delete, and DecreaseKey. The function insert should return the node 
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into which the new element was inserted. This retumed information can 
later be used as an input to Delete and DecreaseKey. 


Prove that if we start with empty F-heaps and perform only the operations 
insert, meld, and delete min, then all min trees in the F-heaps are binomial 
trees. 

Can all the functions on an F-heap be performed in the same amount of 
time using singly linked circular lists rather than doubly linked circular 
lists? (Note that we can delete from an arbitrary node x of a singly linked 
circular list by copying over the data in the next node and then deleting the 
next node rather than the node x.) 


Show that if we start with empty F-heaps and do not perform cascading 
cuts, then it is possible for a sequence of F-heap operations to result in 
degree-k min trees that have only k +1 nodes, k 2 1. 

Is the height of every tree in a Fibonacci heap that has a elements 
log n)? If not, what is the worst-case height as a function of n? 


Suppose we change the rule for a cascading cut so that such a cut is per- 
formed only when a node loses a third child rather than when it loses a 
second child. For this, the chifdCut data member is changed so that it can 
have the values 0, 1, and 2. When a node acquires a new parent, its child- 
Cut data member is set to 1. Each time a node has a child cut off (during a 
delete or decrease-key operation), its childCut data member is increased by 
one (unless this data member is already two). If the childCut data member 
is already two, a cascading cut is performed. 


(a) Obtain a recurrence equation for Nj, the minimum number of nodes 
in a min tree with degree i. Assume that we start with an empty F- 
heap and that all operations (except cascading cut) are performed as 
described in the text. Cascading cuts are performed as described 
above. 


(b) Solve the recurrence of part (a) to obtain a lower bound on N,. 

{c) Does the modified rule for cascading cuts ensure that the minimum 
number of nodes in any min tree of degree i is exponential in i? 

(d) For the new cascading-cut rule, can you establish the same amortized 
complexities as for the original rule? Prove the correctness of your 
answer. 

(e) Answer parts (c) and (d) under the assumption that cascading cuts 
are performed only after k children of a node have been cut off. 
Here, k is a fixed constant (k = 2 for the mule used in the text, and k = 
3 for the rule used earlier in this exercise). 

(f) How do you expect the performance of F-heaps to change as larger 
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values of k (see part (e)) are used? 


7. For the Fibonacci numbers F;, and the numbers N; of Lemma 9.4, prove the 
following: 


h-2 

@ Fy= SR +laok 

k=0 
(b) Use (a) to show that N; = F; 42,120. 
(c) Use the equality 
1 A+y5 
Fy =— -—> 
ar ¢ 2 ¥ 15 ¢ 
to show that F; 4. 2 o*, k 20, where = (1+¥5\/2. 

8. Implement the single-source shortest-path algorithm of Chapter 6 using the 
data structures recommended there as well as F-heaps. However, use adja- 
cency lists rather than an adjacency matrix. Generate 10 connected 
undirected graphs with different edge densities (say 10%, 20%, ---, 100% 
of maximum) for each of the cases n = 100, 200, ---, 500. Assign random 
costs to the edges (use a uniform random number generator in the range [1, 


1000]). Measure the run times of the two implementations of the shortest- 
path algorithms. Plot the average times for each n. 


¥k20 


7 


9.5 PAIRING HEAPS 
9.5.1 Definition 


The pairing heap supports the same operations as supported by the Fibonacci 
heap. Pairing heaps come in two varieties—min pairing heaps and max pairing 
heaps. Min pairing heaps are used when we wish to represent a min priority 
queue, and max pairing heaps are used for max priority queues. In keeping with 
our discussion of Fibonacci heaps, we explicitly discuss min pairing heaps only. 
Max pairing heaps are analogous. Figure 9.14 compares the actual and amor- 
tized complexities of the Fibonacci and pairing heap operations. 

Although the amortized complexities given in Figure 9.14 for pairing heap 
operations are not known to be tight (i.e., no one knows of an operation sequence 
whose run time actually grows logarithmically with the number of decrease key 
operations (say)), it is known that the amortized complexity of the decrease key 
operation is Q(loglogn) (see the section titled References and Selected Readings 
at the end of this chapter). 

Although the amortized complexity is better when a Fibonacci heap is used 
rather than when a pairing heap is used, extensive experimental studies employ- 
ing these structures in the implementation of Dijkstra’s shortest paths algorithm 
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Operation | Pairing Heap 

| Amortized | 
GetMin Ol) 
Insert OW) 
DeleteMin O(n) O(log 2) O(a) Oflogn) | 
Meld ol) O(1) O(1) Oflog x) i 
Delete O(n) O(log n) O(n) Otlog n) 
DecreaseKey | O(n) OW) Ol) O(log n) 


Figure 9.14: Complexity of Fibonacci and pairing heap operations 


(Section 6.4.1) and Prim’s minimum cost spanning tree algorithm (Section 6.3.2) 
indicate that pairing heaps actually outperform Fibonacci heaps. 


Definition: A min pairing heap is a min tree in which the operations are per- 
formed in a manner to be specified later. 


Figure 9.15 shows four example min pairing heaps. Notice that a pairing 
heap is a single tree, which need not be a binary tree. The min element is in the 
root of this tree and hence this element may be found in O(1) time. 


© 


{a) 


3) oO. : 
® 66H Dd © 


{b) () (d) 


Figure 9.15: Example min pairing heaps 
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9.5.2 Meld and Insert 


Two min pairing heaps may be melded into a single min pairing heap by per- 
forming a compare-link operation. In a compare-link, the roots of the two min 
trees are compared and the min tree that has the larger root is made the leftmost 
subtree of the other tree (ties are broken arbitrarily). 

To meld the min trees of Figures 9.15 (a) and (b), we compare the two 
roots. Since tree (a) has the larger root, this tree becomes the leftmost subtree of 
uee (b). Figure 9.16 (a) is the resulting pairing heap. Figure 9.16 (b) shows the 
result of melding the pairing heaps of Figures 9.15 (c) and (d). When we meld 
the pairing heaps of Figures 9.16 (a) and (b), the result is the pairing heap of Fig- 
ure 9.17. 


2) oO 
© © © OR OREOREC 
® ©® © OM © 


(a) Meld of Figures 9.15 (a) and (b) —_(b) Meld of Figures 9.15 (c) and (d) 


Figure 9.16: Melding pairing heaps 


To insert an element x into a pairing heap p, we first create a pairing heap q 
with the single element x, and then meld the two pairing heaps p and q. 


9.5.3 Decrease Key 


Suppose we decrease the key/priority of the element in node N. When N is the 
toot or when the new key in NV is greater than or equal to that in its parent, no 
additional work is to be done. However, when the new key in NV is less than that 
in its parent, the min tree property is violated and corrective action is to be taken. 
For example, if the key in the root of the tree of Figure 9.15 (c) is decreased from 
1 to 0, or when the key in the leftmost child of the root of Figure 9.15 (c) is 
decreased from 4 to 2 no additional work is necessary. When the key in the left- 
most child of the root of Figure 9.15 (c) is decreased from 4 to 0 the new value is 
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Figure 9.17: Meld of Figures 9.16 (a) and (b) 


less than that in the root (see Figure 9.18 (a)) and corrective action is needed. 


QR © 
© © © (5) 
O © ©) 


(a) (b) 


Figure 9.18: Decreasing a key 


Since pairing heaps are normally not implemented with a parent pointer, it 
is difficult to determine whether or not corrective action is needed following a 
key reduction. Therefore, corrective action is taken regardless of whether or not 
it is needed except when N is the tree root. The corrective action consists of the 
following steps: 
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Step 1; Remove the subtree with root NV from the tree. This results in two min 
trees, 


Step 2: Meld the two min trees together. 


Figure 9.18 (b) shows the two min trees following Step 1, and Figure 9.18 
(c) shows the result following Step 2. 


9.5.4 Delete Min 


The min element is in the root of the tree. So, to delete the min element, we first 
delete the root node. When the root is deleted, we are left with zero or more min 
trees (i.¢., the subtrees of the deleted root). When the number of remaining min 
trees is two or more, these min trees must be melded into a single min tree. In 
two pass pairing heaps, this melding is done as follows: 


Step 1: Make a left to right pass over the trees, melding pairs of trees. 
Step 2: Start with the rightmost tree and meld the remaining trees (right to left) 
into this tree one at a time. 


Consider the min pairing heap of Figure 9.19 (a). When the root is 
removed, we get the collection of 6 min trees shown in Figure 9.19 (b). 


eee 


(a) A min pairing heap (b) After deletion of root 


Figure 9.19: Deleting the min element 


In the left to right pass of Step 1, we first meld the trees with roots 4 and 0. 
Next, the trees with roots 3 and 5 are melded. Finally, the tees with roots 1 and 
6 are melded. Figure 9.20 shows the resulting three min trees. 
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0) D 
® @ Oo ® © @ 
® ® 


Figure 9.20: Trees following first pass 


In Step 2 (which is a right to left pass), the two rightmost trees of Figure 
9.20 are first melded to get the tree of Figure 9.21 (a). 


Figure 9.21: First stage of second pass 


Then the tree of Figure 9.20 with root 0 is melded with the tee of Figure 
9.21 to get the final min tree, which is shown in Figure 9.22. 

Note that if the original pairing heap had 8 subtrees, then following the left 
to right melding pass we would be left with 4 min trees. In the right to left pass, 
we would first meld trees 3 and 4 to get tree 5. Then trees 2 and 5 would be 
melded to get tree 6. Finally, we would meld trees | and 6. 

In multi pass pairing heaps, the min trees that remain following the remo- 
val of the root are melded into a single min tree as follows: 


Step 1; Put the min trees onto a FIFO queue. 
Step 2: Extract two trees from the front of the queue, meld them and put the 
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Figure 9.22: Final min pairing heaping following a delete min 


resulting tree at the end of the queue. Repeat this step until only one 
tree remains. 


Consider the six trees of Figure 9.19 (b) that result when the root of Figure 
9.19 (a) is deleted. First, we meld the trees with roots 4 and 0 and put the result- 
ing min tree at the end of the queue. Next, the trees with roots 3 and 5 are 
melded and the resulting min tree is put at the end of the queue. And then, the 
trees with roots 1 and 6 are melded and the resulting min tree added to the queue 
end. The queue now contains the three min trees shown in Figure 9.20. Next, 
the min trees with roots 0 and 3 are melded and the result put at the end of the 
queue. We are now left with the two min trees shown in Figure 9.23. 

Finally, the two min trees of Figure 9.23 are melded to get the min tree of 
Figure 9.24, 


9.5.5 Arbitrary Delete 


Deletion from an arbitrary node A is handled as a delete-min operation when N is 
the root of the pairing heap. When A is not the tree root, the deletion is done as 
follows: 


Step I: Detach the subtree with root W from the tree. 


Step 2: Delete node N and meld its subtrees into a single min tree using the two 
pass scheme if we are implementing a two pass pairing heap or the multi 
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Figure 9.23: Next to last state in multi pass delete 


Figure 9.24: Result of multi pass delete min 

pass scheme if we are implementing a multi pass pairing heap. 
Step 3: Mcld the min trees from Steps 1 and 2 into a single min tree. 
9.5.6 Implementation Considerations 


Although we can implement a pairing heap using nodes that have a variable 
number of children fields, such an implementation is expensive because of the 
need to dynamically increase the number of children fields as needed. An 
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efficient implementation results when we use the binary tree representation of a 
tee (see Section 5.1.2.2). Siblings in the original min tree are linked together 
using a doubly linked list. In addition to a data field, each node has the three 
pointer fields previous, next, and child. The leftmost node in a doubly linked hist 
of siblings uses its previous pointer to point to its parent. A leftmost child 
satisfies the property x -» previous > child = x. The doubly linked list makes it 
is possible to remove an arbitrary element (as is required by the Delete and 
DecreaseKey operations) in O(1) time. 


9.5.7 Complexity 


You can verify that using the described binary tree representation, all pairing 

heap operations (other than Delete and DeleteMin) can be done in O(1) time. 

The complexity of the Delete and DeleteMin operations is O(n), because the 

ae of subtrees that have to be melded following the removal of a node is 
in). 

The amortized complexity of the pairing heap operations is established in 
the paper by Fredman et al. cited in the References and Selected Readings sec- 
tion. Experimental studies conducted by Stasko and Vitter (see their paper that 
is cited in the References and Selected Readings section) establish the superior- 
ity of two pass pairing heaps over multipass pairing heaps. 


EXERCISES 


1, (a) Into an empty two pass min pairing heap, insert elements with Priori- 
ties 20, 10, 5, 18, 6, 12, 14, 9, 8 and 22 (in this order). Show the min 
pairing heap following each insert. 

(b) Delete the min element from the final min pairing heap of part (a). 
Show the resulting pairing heap. 

2. {a) Into an empty multi pass min pairing heap, insert elements with 
priorities 20, 10, 5, 18, 6, 12, 14, 9, 8 and 22 (in this order). Show the 
min pairing heap following each insert. 

{b) Delete the min element from the final min pairing heap of part (a). 
Show the resulting pairing heap. 

3. Fully code and test the class MuitiPassPairingHeap, which impements a 
multi pass min pairing heap. Your class must include the functions Ger- 
Min, Insert, DeleteMin, Meld, Delete and DecreaseKey. The function 
Insert should retum the node into which the new element was inserted. 


This returned information can later be used as an input to Delete and 
DecreaseKey. 
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4. What are the worst-case height and degree of a pairing heap that has n ele- 
ments? Show how you arrived at your answer. 

5. Define a one puss pairing heap as an adaptation of a two pass pairing heap 
in which Step 1 (Make a left to right pass over the trees, melding pairs of 
trees.) is eliminated. Show that the amortized cost of cither insert or delete 
min must be O(n). 


9.6 SYMMETRIC MIN-MAX HEAPS 


9.6.1 Definition and Properties 


A double-ended priority queue (DEPQ) may be represented using a symmetric 
min-max heap (SMMH). An SMMH is a complete binary wee in which each 
node other than the root has exactly one element. The root of an SMMH is empty 
and the total number of nodes in the SMMH is n +1, where n is the number of 
elements. Let N be any node of the SMMH. Let elements (N) be the elements in 
the subtree rooted at NV but excluding the element (if any) in N. Assume that 
elements (N)#0. N satisfies the following properties: 


Ql: The teft child of N has the minimum element in elements (N). 
Q2: The right child of N (if any) has the maximum element in elements (N). 


Figure 9.25 shows an example SMMH that has 12 elements, When N 
denotes the node with 80, elements (N)={6,14,30,40}; the left child of N has the 
minimum element 6 in elements (N); and the right child of N has the maximum 
element 40 in elements (N). You may verify that every node N of this SMMH 
satisfies properties QI and Q2. 

It is easy to see that an n +1-node complete binary tree with an empty root 
and one element in every other node is an SMMH iff the following are true: 


Pl: The element in each node is less than or equal to that in its right sibling (if 
any). 

P2: Forevery node N that has a grandparent, the element in the left child of the 
grandparent is less than or equal to that in NV. 


P3: For every node N that has a grandparent, the element in the right child of 
the grandparent is greater than or equal to that in N. 


Properties P2 and P3, respectively, state that the grandchildren of each 
node M have elements that are greater than or equal to that in the left child of M 
and less than or equal to that in the right child of M. Hence, P2 and P3 follow 
from QI and Q2, respectively. Notice that if property PI is satisfied, then at 
most one of P2 and P3 may be violated at any node N. Using properties Pi 


530 Priority Queues 


Figure 9.25: A symmetric min-max heap 


through P3 we arrive at simple algorithms to insert and delete elements. These 
algorithms are simple adaptations of the corresponding algorithms for heaps. 

As we shall see, the standard DEPQ operations of Program 9.1 can be done 
efficiently using an SMMH. 


9.6.2 SMMH Representation 


Since an SMMH is a complete binary tree, it is efficiently represented as a one- 
dimensional array (say h) using the standard mapping of a complete binary tree 
into an array (Section 5.2.3.1). Position 0 of h is not used and position 1, which 
represents the root of the complete binary tree, is empty. We use the variable 
last to denote the rightmost position of h in which we have stored an element of 
the SMMH. So, the size (i.e., number of elements) of the SMMH is Jast-1. The 
variable arrayLength keeps track of the current number of positions in the array 
A, 


When n=l, the minimum and maximum elements are the same and are in 
the left child of the root of the SMMH. When »>1, the minimum element is in 
the left child of the root and the maximum is in the right child of the root. So, the 
GetMin and GetMax operations take O(1) time each. Program 9.7 defines the 
class SMMH, which implements a symmetric min-max heap. Program 9.8 gives 
the constructor that creates an empty SMMH. 
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template <class T> 

class SMMH : public DEPQ { 

public: 
SMMH(int initialCapacity = 10); 
“SMMH() {delete (] 43} 


const T& GetMin() const 

{// return min element 
if (/ast == 1) throw QueueEmpty(); 
return }(2); 


} 


const T& GetMax{) const 

{/ return max element 
if (last == 1) throw QueueEmpty(); 
if (ast == 2) return A(2]; 
else return /{3}; 


void Insert(const T&); 
void DeleteMin(); 
void DeleteMax(); 
private: 
int last; 1 position of last element in queue 
int arrayLength; // queue capacity +2 
T *h; H element array 


' 


Program 9.7: Class definition for symmetric min-max heap 
9.6.3 Inserting into an SMMH 


The algorithm to insert into an SMMH has three steps. 


Step 1: Expand the size of the complete binary tree by 1, creating a new node E 
for the element x that is to be inserted. This newly created node of the 
complete binary tree becomes the candidate node to insert the new ele- 
ment x. 

Step 2: Verify whether the insertion of x into E would result in a violation of 
property P1. Note that this violation occurs iff E is a right child of its 
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template <class T> 
SMMH<T>::SMMH (int initialCapacity) 
{4 Constructor. 

if (initialCapacity < 1) 

{ 


ostringstream s; 
5 << “Initial capacity =" << initialCapacity <<" Must be > 0"; 
throw /legalParameterValue(s.str); 

} 

arrayLength = initialCapacity + 2; 

h= new TlarrayLength); 

last =1; 


} 
Program 9.8 Constructor for SMMH 


parent and x is greater than the element in the sibling of E. In case of a 
P1 violation, the element in the sibling of E is moved to E and E is 
updated to be the now empty sibling. 


Step 3: Perform a bubble-up pass from E up the tree verifying properties P2 and 
P3. In each round of the bubble-up pass, E moves up the tree by one 
level. When E is positioned so that the insertion of x into E doesn’t 
result in a violation of either P2 or P3, insert x into E. 

Suppose we wish to insert 2 into the SMMH of Figure 9.25. Since an 
SMMH is a complete binary tree, we must add a new node to the SMMH in the 
position shown in Figure 9.26; the new node is labeled £. In our example, E will 
denote an empty node. 

If the new element 2 is placed in node E, property P2 is violated as the left 
child of the grandparent of E has 6. So we move the 6 down to E and move E up 
one level to obtain the configuration of Figure 9.27. 

Now we determine if it is safe to insert the 2 into node E. We first notice 
that such an insertion cannot result in a violation of property PI, because the pre- 
vious occupant of node E was greater than 2. For properties P2 and P3, let N=E. 
P3 cannot be violated for this value of N as the previous occupant of this node 
was greater than 2. So, only P2 can be violated. Checking P2 with N=E, we see 
that P2 will be violated if we insert x=2 into E, because the left child of the 
grandparent of E has the element 4. So we move the 4 down to E and move E up 


one level to the node that previously contained the 4. Figure 9.28 shows the 
resulting configuration. 
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Figure 9.26: The SMMH of Figure 9.25 with a node added 


6 


® &® 


Figure 9.27: The SMMH of Figure 9.26 with 6 moved down 


For the configuration of Figure 9.28 we see that placing 2 into node & can- 
not violate property Pl, because the previous occupant of node E was greater 
than 2. Also properties P2 and P3 cannot be violated, because node & has no 
grandparent. So we insert 2 into node E and obtain Figure 9.29. 

Let us now insert 50 into the SMMH of Figure 9.29. Since an SMMH is a 
complete binary tree, the new node must be positioned as in Figure 9.30. 

Since E is the right child of its parent, we first check Pl at node E. If the 
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Figure 9.28: The SMMH of Figure 9.27 with 4 moved down 


Figure 9.29: The SMMH of Figure 9.28 with 2 inserted 


new element (in this case 50) is smaller than that in the left sibling of E, we swap 
the new element and the element in the left sibling. In our case, no swap is done. 
Then we check P2 and P3. We see that placing 50 into E would violate P3. So 
the element 40 in the right child of the grandparent of E is moved down to node 
E. Figure 9.31 shows the resulting configuration. Placing 50 into node E of Fig- 
ure 9.31 cannot create a P1 violation because the previous occupant of node E 
was smaller, A P2 violation isn't possible either. So only P3 needs to be checked 
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Figure 9,30: The SMMH of Figure 9.29 with a node added 


at E, Since there is no P3 violation at E, 50 is placed into E. 


Figure 9.31: The SMMH of Figure 9.30 with 40 moved down 


Program 9.9 gives the C++ code for the insert operation; the variable 
currentNode refers to the empty node E of our example. Since the height of a 
complete binary tree is O(log n) and Program 9.9 does O(1) work at each level 
of the SMMH, the complexity of the insert function is O(log n). 
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template <class T> 
void SMMH<T>::insert(const T& x) 
{// Insert x into the SMMH. 
# increase array length if necessary 
if (last == arrayLength - 1) 
{/ double array length 
ChangeSize! D(h, arrayLength, 2 * arrayLength); 
arrayLength *= 2; 


find place for x 
4 currentNode starts at new leaf and moves up tree 
int currentNode = ++last; 
if (last % 2 == 1 && x < h{last —1)) 
{// left sibling must be smaller, P1 
Allast] = h{last - 1}; 
currentNode——; 


bool done = false; 
while (done && currentNode >= 4) 
{i/ currentNode has a grandparent 
int gp = currentNode /4; / grandparent 
int legp = 2 * gp; MH left child of gp 
int regp =Icgp + 1; # right child of gp 
if (x < Aicgp)) 
{/ P2 is violated 
hicurrentNode) = h{Icgp); 
currentNode = Icgp; 


} 
else if (x > Alrcgp]) 
{// P3 is violated 
A{currentNode] = h[regp); 
currentNode = rcgp; 


} 


else done =true; // neither P2 nor P3 violated 


hlcurrentNode] = x; 


} 


Program 9.9: Insertion into a symmetric min-max heap 
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9.6.4 Deleting from an SMMH 


The algorithm to delete either the min or max element is an adaptation of the 
trickle-down algorithm used to delete an element from a min or a max heap. We 
consider only the case when the minimum element is to be deleted. If the SMMH 
is empty, the deletion cannot be performed. So, assume we have a non-empty 
SMMH. The minimum element is in A [2]. If fast=2, the SMMH becomes empty 
following the deletion. Assume that /ast#2, Let x=h [last] and decrement last by 
1, To complete the deletion, we must reinsert x into an SMMH whose / [2] node 
is empty. Let E denote the empty node. We follow a path from E down the tree, 
as in the delete algorithm for a min or max heap, verifying properties P1 and P2 
until we reach a suitable node into which x may be inserted. In the case of a 
delete-min operation, the trickle-down process cannot cause a P3 violation. So, 
we don't explicitly verify P3. 

Consider the SMMH of Figure 9.31 with 50 in the node labeled E. A 
delete min results in the removal of 2 from the left child of the root (i.¢., h(2)) 
and the removal of the last node (i.e., the one with 40) from the SMMH. So, 
x=40 and we have the configuration shown in Figure 9.32. Since h[3) has the 
maximum element, P1 cannot be violated at £. Further, since Eis the left child of 
its parent, no P3 violations can result from inserting x into £. So we need only 
concern ourselves with P2 violations. To detect such a violation, we determine 
the smaller of the left child of E and the left child of £’s right sibling. For our 
example, the smaller of 8 and 4 is determined. This smaller element 4 is, by 
definition of an SMMH, the smallest element in the SMMH. Since 4<x=40, 
inserting x into E would result in a P2 violation. To avoid this, we move the 4 
into node E and the node previously occupied by 4 becomes E (see Figure 9.33). 
Notice that if 42x, inserting x into E would result in a properly structured 
SMMH. 

Now the new E becomes the candidate node for the insertion of x. First, we 
check for a possible P1 violation that may result from such an insertion. Since 
x=40<50, no PI violation results. Then we check for a P2 violation. The left 
children of E and its sibling are 14 and 6. The smaller child, 6, is smaller than x. 
So, x cannot be inserted into E. Rather, we swap E and 6 to get the configuration 
of Figure 9,34, 

We now check the P1 property at the new E. Since E doesn’t have a right 
sibling, there is no PI violation. We proceed to check the P2 property. Since E 
has no children, a P2 violation isn’t possible either. So, x is inserted into E. Let's 
consider another delete-min operation. This time, we delete the minimum ele- 
ment from the SMMH of Figure 9.34 (recall that the node labeled E contains 40). 
The min element 4 is removed from h(2] and the last element, 40, is removed 
from the SMMH and placed in x. Figure 9.35 shows the resulting configuration. 

As before, a P1 violation isn’t possible at h{2]. The smaller of the left 
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x=40 


Figure 9.32: The SMMH of Figure 9.31 with 2 deleted 


Figure 9.33: The SMMH of Figure 9.32 with E and 4 interchanged 


children of E and its sibling is 6. Since 6<x=40, we interchange 6 and E to get 
Figure 9.36. 


Next, we check for a possible P1 violation at the new E. Since the sibling 
of E is 50 and x=40 < 50, no P1 violation is detected. The smaller left child of E 


Interval Heaps 539 


é 


x=40 
Figure 9.34: The SMMH of Figure 9.33 with £ and 6 interchanged 


x= 


Figure 9.35: First step of another delete min 


and its sibling is 14 (actually, the sibling doesn’t have a left child, so we just use 
the left child of £), which is <x. So, we swap E and 14 to get Figure 9.37. 

Since there is a P1 violation at the new E, we swap x and 30 to get Figure 
9.38 and proceed to check for a P2 violation at the new E. As there is no P2 vio- 
jation here, x=30 is inserted into E. 
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Figure 9.36: The SMMH of Figure 9.35 with E and 6 interchanged 


Figure 9.37: The SMMH of Figure 9.36 with E and 14 interchanged 


We leave the development of the code for the delete operations as an exer- 
cise. However, you should note that these operations spend O(1) time per level 
during the trickle-down pass. So, their complexity is O(log 2). 
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x=30 


Figure 9.38: The SMMH of Figure 9.37 with x and 30 interchanged 


EXERCISES 


be 


2; 


9.7 


9.7.1 


Show that every complete binary tree with an empty root and one element 
in every other node is an SMMH iff P1 through P3 are true. 

Start with an empty SMMH and insert the elements 20, 10, 40, 3, 2, 7, 60, 1 
and 80 (in this order) using the insertion algorithm developed in this sec- 
tion. Draw the SMMH following each insert. 

Perform 3 delete-min operations on the SMMH of Figure 9.38 with 30 in 
the node E. Use the delete min strategy described in this section. Draw the 
SMMH following each delete min. 

Perform 4 delete max operations on the SMMH of Figure 9.38 with 30 in 
the node E. Adapt the delete min strategy of this section to the delete max 
operation. Draw the SMMH following each delete max operation. 
Develop the code for all functions of the class SMMH (Program 9.7). Test 
all functions using your own test data. 


INTERVAL HEAPS 


Definition and Properties 


Like an SMMH, an interval heap is a heap inspired data structure that may be 
used to represent a DEPQ. An interval heap is a complete binary tree in which 
each node, except possibly the last one (the nodes of the complete binary tree are 
ordered using a level order traversal), contains two elements. Let the two 
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elements in a node be a and b, where a $b. We say that the node represents the 
closed interval [a,b]. @ is the left end point of the node’s interval and b is its 
right end point. 

The interval [c,d] is contained in the interval [a,b] iffasc<sdsb. Inan 
interval heap, the intervals represented by the left and right children (if they 
exist) of each node P are contained in the interval represented by P. When the 
fast node contains a single element c, then a $c <b, where [a,b] is the interval 
of the parent (if any) of the last node. 

Figure 9.39 shows an interval heap with 26 elements. You may verify that 
the intervals represented by the children of any node P are contained in the inter- 
val of P. 


Figure 9.39: An interval heap 


The following facts are immediate: 


(1) The left end points of the node intervals define a min heap, and the right 
end points define a max heap. In case the number of elements is odd, the 
last node has a single element which may be regarded as a member of 
either the min or max heap. Figure 9.40 shows the min and max heaps 
detined by the interval heap of Figure 9.39. 

(2) When the root has two elements, the left end point of the root is the 
minimum element in the interval heap and the right end point is the max- 
imum. When the root has only one element, the interval heap contains just 
‘one element. This element is both the minimum and maximum element. 


(3) An interval heap can be represented compactly by mapping into an array as 
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(b) max heap 


(a) min heap 
Figure 9.40: Min and max heaps embedded in Figure 9.39 


is done for ordinary heaps. However, now, each array position must have 


space for two elements. 
(4) The height of an interval heap with n elements is O(log 7). 


9.7.2 Inserting into an Interval Heap 


Suppose we are to insert an element into the interval heap of Figure 9.39. Since 
this heap currently has an even number of elements, the heap following the inser- 
tion will have an additional node A as is shown in Figure 9.41. a 
The interval for the parent of the new node A is [6,15]. Therefore, hed 
new element is between 6 and 15, the new element may be inserted into node A: 
When the new element is less than the left end point 6 of the parent intery ae 
new element is inserted into the min heap embedded in the i ape ‘ 
insertion is done using the min heap insertion Lismeg prey aa inter- 
When the new element is greater than the right end point 1 fod 5 iy interval 
val, the new element is inserted into the max heap eee starting at 
heap. This insertion is done using the max heap insertion Pr 
node A. . 5 ap of Figure 9.39, this 
If we are to insert the element 10 into the ete cea 3. we 


element is put into the node A shown in eat left end points down until 
nen nd point is $3. The new 


follow a path from node A towards the pia 
we either pass the root or reach a node whose 
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Figure 9.41: Interval heap of Figure 9.39 after one node is added 


element is inserted into the node that now has no left end point. Figure 9.42 
shows the resulting interval heap. 


3,17 


(10 {5,11} (39) 47 
ROP 


Figure 9.42: The interval heap of Figure 9.39 with 3 inserted 


To insert the element 40 into the interval heap of Figure 9.39, we follow a 
path from node A (see Figure 9.41) towards the root, moving right end points 
down until we either pass the root or reach a node whose right end point is 2 40. 
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The new element is inserted into the node that now has no right end point. Figure 
9.43 shows the resulting interval heap. 


Figure 9.43: The interval heap of Figure 9.39 with 40 inserted 


9.4 Now, Suppose we wish to insert an element into the interval heap of Figure 

.43. Since this interval heap has an odd number of elements, the insertion of the 
new element does not increase the number of nodes. The insertion procedure is 
the same as for the case when we initially have an even number of elements. Let 
A denote the last node in the heap. If the new element lies within the interval 
[6,15] of the parent of A, then the new element is inserted into node A (the new 
element becomes the left end point of A if it is less than the element currently in 
A). If the new element is less than the left end point 6 of the parent of A, then the 
new element is inserted into the embedded min heap; otherwise, the new element 
iS inserted into the embedded max heap. Figure 9.44 shows the result of insert- 
ing the element 32 into the interval heap of Figure 9.43. 


9.7.3 Deleting the Min Element 


The removal of the minimum element is handled as several cases: 


When the interval heap is empty. the DelereMin operation fails. 
When the interval heap has only one element, this element is the element to 
be returned. We leave behind an empty interval heap. 


a) 
(2) 
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Figure 9.44: The interval heap of Figure 9.43 with 32 inserted 


(3) When there is more than one element, the left end point of the root is to be 
returned. This point is removed from the root. If the root is the last node of 
the interval heap, nothing more is to be done. When the last node is not the 
root node, we remove the left point p from the last node. If this causes the 
last node to become empty, the last node is no longer part of the heap. The 
point p removed from the last node is reinserted into the embedded min 
heap by beginning at the root. As we move down, it may be necessary to 
swap the current p with the right end point r of the node being examined to 
ensure that p Sr. The reinsertion is done using the same strategy as used 
to reinsert into an ordinary heap. 


Let us remove the minimum element from the interval heap of Figure 9.44. 
First, the element 2 is removed from the root. Next, the left end point 15 is 
removed from the last node and we begin the reinsertion procedure at the root. 
The smaller of the min heap elements that are the children of the root is 3. Since 
this element is smaller than 15, we move the 3 into the root (the 3 becomes the 
left end point of the root) and position ourselves at the left child B of the root. 
Since, 15 < 17 we do not swap the right end point of B with the current p=15. 
The smaller of the left end points of the children of B is 3. The 3 is moved from 
node C into node B as its left end point and we position ourselves at node C. 
Since p=15>11, we swap the two and 15 becomes the right end point of node c 
The smaller of left end points of Cs children is 4. Since this is smaller than the 
current p=11, it is moved into node C as this node’s left end point. We now 


Interval Heaps $47 


position ourselves at node D. First, we swap p=11 and Ds right end point. Now, 
since D has no children, the current p=7 is inserted into node D as Ds left end 


point. Figure 9.45 shows the result. 


Figure 9,45. * The interval heap of Figure 9.44 with minimum element removed 


The max element may be removed using an analogous procedure. 


9.7.4 Initializing an Interval Heap 


Interval heaps may be initialized using a strategy similar to that used to initialize 

seh AY heaps—work your way from the heap bottom to the root ensuring that 
each subtree is an interval heap. For each subtree, first order the elements in the 
Fook; then reinsert the left end point of this subtree's root using the reinsertion 
steBy used for the DeleteMin operation, then reinsert the right end point of this 
Subtree’s root using the strategy used for the DeleteMax operation. 


9.7.5 Complexity of Interval Heap Operations 
The operations Ger (Min () and GetMax () take O{!) time each; Insert (x), Delete- 


Min(), and DeleteMax (© take O(log #) each; and initializing an element inter- 
val heap takes O(7) time. 
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In the complementary range search problem, we have a dynamic collection (i.e., 
points are added and removed from the collection as time goes on) of one- 
dimensional points (i.e., points have only an x-coordinate associated with them) 
and we are to answer queries of the form: what are the points outside of the inter- 
val [a,b]? For example, if the point collection is 3,4,5,6,8,12, the points outside 
the range (5,7] are 3,4,8,12. 

‘When an interval heap is used to represent the point collection, a new point 
can be inserted or an old one removed in O(log 7) time, where n is the number of 
points in the collection. Note that given the location of an arbitrary element in 
an interval heap, this element can be removed from the interval heap in O(log n) 
time using an algorithm similar to that used to remove an arbitrary element from 
a heap. 

The complementary range query can be answered in O(k) time, where is 
the number of points outside the range (a,b). This is done using the following 
recursive procedure: 


Step 1: If the interval tee is empty, return. 


Step 2: If the root interval is contained in [a,b], then all points are in the range 
(therefore, there are no points to report), return, 


Step 3: Report the end points of the root interval that are not in the range [a,b]. 


Step 4: Recursively search the left subtree of the root for additional points that 
are not in the range [a,b). 

Step 5: Recursively search the right subtree of the root for additional points that 
are not in the range {a,b}. 

Step 6: return. 


Let us try this procedure on the interval heap of Figure 9.44. The query 
interval is [4,32]. We start at the root. Since the root interval is not contained in 
the query interval, we reach step 3 of the procedure. Whenever step 3 is reached. 
we are assured that at least one of the end points of the root interval is outside 
the query interval. Therefore, each time step 3 is reached, at least one point 1s 
reported. In our example, both points 2 and 40 are outside the query interval and 
are reported. We then search the left and right subtrees of the root for additional 
points. When the left subtree is searched, we again determine that the root inter 
val is not contained in the query interval. This time only one of the root interval 
points (i.e., 3) is outside the query range. This point is reported and we proceed 
to search the left and right subtrees of B for additional points outside the query 
range. Since the interval of the left child of B is contained in the query range. the 
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Jeft subtree of B contains no points outside the query range. We do not explore 
the left subtree of B further. When the right subtree of B is searched, we report 
the left end point 3 of node C and proceed to search the Jeft and right subtrees of 
C. Since the intervals of the roots of each of these subtrees is contained in the 
query interval, these subtrees are not explored further. Finally, we examine the 
root of the right subtree of the overall tree root, that is the node with interval 
{4,32}. Since this node’s interval is contained in the query interval, the right sub- 
tree of the overall tree is not searched further. 

We say that a node is visited if its interval is examined in Step 2. With this 
definition of visited, we see that the complexity of the above six step procedure 
is O(number of nodes visited). The nodes visited in the preceding example are 
the root and its two children, the two children of node B, and the two children of 
node C. So, 7 nodes are visited and a total of 4 points are reported. 

We show that the total number of interval heap nodes visited is at most 
3k+1, where & is the number of points reported. If a visited node reports one or 
two points, give the node a count of one. Ifa visited node reports no points, give 
it a Count of zero and add one to the count of its parent (unless the node is the 
foot and so has no parent). The number of nodes with a nonzero count is at most 
Since no node has a count more than 3, the sum of the counts is at most 3k. 
Accounting for the Possibility that the root reports no point, we see that the 

er of nodes visited is at most 3k +1. Therefore, the complexity of the search 


numb 
is O(k). This complexity is asymptotically optimal because every algorithm that 


Teports & points must spend at least @(1) time per reported point. 
In our example search, the root gets a count of 2 (1 because it is visited and 
reports at least one point and another 1 because its right child is visited but 
Teports no point), node B gets a count of 2 (1 because it is visited and reports at 
least one point and another 1 because its left child is visited but reports no point), 
and node C gets a count of 3 (1 because it is visited and reports at least one point 
and another 2 because its left and right children are visited and neither reports a 
Point). The count for each of the remaining nodes in the interval heap is 0. 


EXERCISES 
1. Start with an emply interval heap and insert the elements 20, 10, 40, 3, 2, 7, 
60. 1 and 80 (in this order) using the insertion algorithm developed in this 
Section. Draw the interval heap following each insert. 
Perform 3 delete-min operations on the interval heap of Figure 9.45. Use 
the delete min strategy described in this section. Draw the interval heap 


v 


following each delete min. 
3. Perform 4 delete max operations on the interval heap of Figure 9.45. Adapt 
the delete min strategy of this section to the delete max operation. Draw 


the intervat heap following each delete max operation. 
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4. Develop the code for al! functions of the class /ntervalHeap, which imple- 
ments the interval heap data structure and derives from the virtual class 
DEPQ. In addition to the functions specified in DEPQ you also must code 
the initialization function and a function for the complementary range 
search operation. Test all functions using your own test data. 


5. The min-max heap is an alternative heap inspired data structure for the 
representation of a DEPQ. A min-max heap is a complete binary tree in 
which each node has exactly one element. Alternating levels of this tree 
are min levels and max levels, respectively. The root is on a min level. Let 
x be any node in a min-max heap. If x is on a min (max) level then the ele- 
ment in x has the minimum (maximum) priority from among all elements in 
the subtree with root x. A node on a min (max) level is called a min (max) 
node. Figure 9.46 shows an example 12-elements min-max heap. We use 
shaded circles for max nodes and unshaded circles for min nodes. 


min 


max 


max 


Figure 9.46: A 12-element min-max heap 


Fully code and test the class MinMaxHeap, which impements a min-max 
heap using a one-dimensional array for the complete binary tree. Your 
class must provide an implementation for all DEPQ functions. The com- 
plexity of GerMin and GetMax should be O(1) and that for the remaining 
DEPQ functions should be O(log 1). 
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Height-biased leftist trees were invented by C. Crane. See, Linear Lists and 
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Complexity of Building an Interval Heap,”’ by Y. Ding and M. Weiss, nforma- 
tion Processing Letters, 50, 143-144, 1994; “Interval heaps,’’ by J. van Leeuwen 
and D. Wood, The Computer Journal, 36, 3, 209-216, 1993; ‘‘A mergeable 
double-ended priority queue,” by S. Olariu, C. Overstreet, and Z. Wen, The 
Computer Journal, 34, 5, 423-427, 1991; and ‘Algorithm 232,” by J. Williams, 
Communications of the ACM, 7, 347-348, 1964. 

The min-max heap and deap are additional heap-inspired stuctures for 
DEPQs. These data structures were developed in ‘‘Min-max heaps and general- 
ized priority queues," by M. Atkinson, J. Sack, N. Santoro, and T. Strothotte, 
Communications of the ACM, 29:10, 1986, pp. 996-1000 and “'The deap: A 
double-ended heap to implement double-ended priority queues,"” by S. Carlsson, 
Information Processing Letters, 26, 1987, pp. 33-36, respectively. 

Data structures for meldable DEPQs are developed in “‘The relaxed min- 
max heap: A mergeable double-ended priority queue,’’ by Y. Ding and M. Weiss, 
Acta Informatica, 30, 215-231, 1993; ‘*Fast meldable priority queues,’’ by G. 
Brodal, Workshop on Algorithms and Data Structures, 1995 and ‘‘Mergeable 
double ended priority queue,” by S. Cho and S. Sahni, international Journal on 
Foundation of Computer Sciences, 10, 1, 1999, 1-18. 

General techniques to arrive at a data structure for a DEPQ from one for a 
single-ended priority queue are developed in **Correspondence based data struc- 
tures for double ended priority queues,’ by K. Chong and S. Sahni, ACM Jr. on 
Experimental Algorithmics, Volume 5, 2000, Article 2. 

For more on priority queues, see Chapters 5 through 8 of ‘‘Handbook of 
data structures and applications,”’ edited by D. Mehta and S. Sabni, Chapman & 
Hall/CRC, Boca Raton, 2005. 
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Efficient Binary Search 
Trees 


10.1 OPTIMAL BINARY SEARCH TREES 


Binary search trees were introduced in Chapter 5. in this section, Th sag pe 
the construction of binary search trees for a static set of elements. That : 
make no additions to or deletions from the set. Only searches are per ome ean 

A sorted list can be searched using a binary search. For a Se aie the 
construct a binary search tree with the property that seareniee aoe on the 
function Ger (Program 5.19) is equivalent to performing a t Hist (5,10. 15) (for 
Sorted list. For instance, a binary search on the sorted element Ha enka ah 
convenience, all examples in this chapter show only an psa iaryeseaieh 
the complete element) corresponds to using algorithm Ger mh ity: nonbethe 
tree of Figure 10.1. Although this tree is a full binary ced ‘i Ra peta he 
optimal binary search tree to use when the probabilities wit 
ments are searched are different. . i . 

To find an api binary search tree for a given collection of elements, we 
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© 
© Gs) 


Migere 10.1: Binary search tree corresponding to a binary search on the list (5, 
, 15) 


must first decide on a cost measure for search trees. When searching for an ele- 
ment at level /, function Ger makes / iterations of the for loop. Since this for 
loop determines the cost of the search, it is reasonable to use the level number of 
anode as its cost. 


Figure 10.2: Two binary search trees 


Example 10.1: Consider the two search trees of Figure 10.2. The second of 
these requires at most three comparisons to decide whether the element being 
sought is in the tree. The first binary tree may require four comparisons, since 
any search key k such that 10 <k < 20 will test four nodes. Thus, as far as 
worst-case search time is concerned, the second binary tree is more desirable 
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than the first. To search for a key in the first tree takes one comparison for the 
10, two for each of 5 and 25, three for 20, and four for 15. Assuming that each 
key is searched for with equal probability, the average number of comparisons 
for a successful search is 2.4. For the second binary search tree this amount is 
2.2, Thus, the second tree has a better average behavior, too. 

Suppose that each of 5, 10, 15, 20 and 25 is searched for with probablility 
0.3, 0.3, 0.05, 0.05 and 0.3, respectively. The average number of comparisons for 
a Successful search in the trees of Figure 10.2 (a) and (b) is 1.85 and 2.05, respec- 
tively. Now, the first tree has better average behavior than the second tree! 0 


In evaluating binary search trees, it is useful to add a special *square’’ 
node at every null link. Doing this to the trees of Figure 10.2 yields the trees of 
Figure 10.3. Remember that every binary tree with n nodes has n + | null links 
and therefore will have n +1 square nodes. We shall call these nodes external 
nodes because they are not part of the original tree. The remaining nodes will be 
called internal nodes. Each time a binary search tee is examined for an 
identifier that is not in the tree, the search terminates at an external node. Since 
all such searches are unsuccessful searches, external nodes will also be referred 
to as failure nodes. A binary wee with external nodes added is an extended 
binary tree. The concept of an extended binary tree as just defined is the same as 
that defined in connection with leftist trees in Chapter 9. Figure 10.3 shows the 
extended binary trees corresponding to the search trees of Figure 10.2. 

We define the external path length of a binary tree to be the sum over all 
external nodes of the lengths of the paths from the root to those nodes. Analo- 
ously, the internal path length is the sum over all internal nodes of the lengths 
Of the paths from the root to those nodes. The internal path length, /, for the ee 
of Figure 10.3(a) is 


T=04+14+14+2+3=7 
Its external path length, E, is 
EB=24+24+44¢4434+2=17 


Exercise 1 of this section shows that the internal and external path lengths 
of a binary tree with n internal nodes are related by the formula E = / + 2n. 
Hence, binary trees with the maximum E also have maximum /. Over all binary 
trees with n internal nodes, what are the maximum and minimum possible values 
for /? The worst case, clearly, is when the tree is skewed (i.e., when the tree has 
a depth of n). In this case, 


ri a(n- 12 
i) 
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(a) 


Figure 10.3: Extended binary trees corresponding to search trees of Figure 10.2 


To obtain trees with minimal /, we must have as many internal nodes as 
close to the root as possible. We can have at most 2 nodes at distance 1, 4 at dis- 
tance 2, and in general, the smallest value for J is 


O+2* 1444248434 --- + 


One tree with minimal internal path length is the complete binary tree 
defined in Section 5.2. If we number the nodes in a complete binary tree as in 
Section 5.2, then we see that the distance of node i from the root is | logzi J. 
Hence, the smallest value for / is 


> [logzi] = Oflogan) 


Isisn 


Let us now retum to our original problem of representing a static element 
set as a binary search tree. Let aj, @2, -**,@, witha) <az< -+* <a, be the 
clement keys. Suppose that the probability of searching for each a; is pj. The 
total cost of any binary search tree for this set of keys is 


L pr levelfa,) 
Isrs 
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when only successful searches are made. Since unsuccessful searches (ie, 
Searches for keys not in the table) will also be made, we should include the cost 
Of these searches in our cost measure, too. Unsuccessful searches terminate with 
algorithm Ger returning a 0 pointer (Program 5.19). Every node with an emply 
Subtree defines a point at which such a termination can take place. Let us 
replace every empty subtree by a failure node. The keys not in the binary search 
(ee may be partitioned into 2 + 1 classes E;, OSiSn. Eg contains all keys X 
Such that X < ay. E; contains all keys X such that a; <X <aj4,, 1 Si<n, and 
E,, contains all keys X, X >a, It is easy to see that for ali keys in a particular 
class Ej, the search terminates at the same failure node, and it terminates at 
different failure nodes for keys in different classes. The failure nodes may be 
numbered 0 to n, with i being the failure node for class Ej, OS iS. If g; is the 
Probability that the key being sought is in £,, then the cost of the failure nodes is 


D gir (level(failure node i) — 1) 
Osisn 


Therefore, the total cost of a binary search tree is 


DX Pi level(a;)+ > gj: (level (failure node i) - 1) (10.1) 
Isis Osisn 


An optimal binary search tree for a1, *~*, pq is one that minimizes Eq. (10-1) 
Over all possible binary search trees for this set of keys. Note that since all 
searches must terminate either successfully or unsuccessfully, we have 


Let Daal 
Isisn—OSiSn 
Example 10.2: Figure 10.4 shows the possible binary search trees for the key set 


be, hee (5, 10, 15). With equal probabilities, p; = qj = 7 for all i and j, 


cost (tree a) = 15/7; cost (tree b) = 13/7 

cost (tree c) = 15/7; cost (tree d) = 15/7 

cost (tree e) = 15/7 
As expected, tree b is optimal. With py = 0.5. p> = 0.1, p3 = 0.05. go = 0.15, 
91 = 0.1, 42 = 0.05, and 3 = 0.05 we have 

cost (tree a) = 2.65; cost (tree b) = 1.9 


cost (tree c} = 1.5; cost (tree d} = 2.05 
cost {tree 2) = 1.6 


Tree ¢ is optimal with this assignment of p's and q’s. 0 
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(b) 


(a) 


(d) (e) 
Figure 10.4: Binary search trees with three elements 


How does one determine the optimal binary search tree? We could proceed 
as in Example 10.2 and explicitly generate all possible binary search trees, then 
compute the cost of each tree, and determine the tree with minimum cost. Since 
the cost of an n-node binary search tree can be determined in O(1) time, the com- 
plexity of the optimal binary search tree algorithm is O(n N(m)), where N{n) is 
the number of distinct binary search trees with n keys. From Section 5.11 we 
know that N(n) = O(4"/n~*). Hence, this brute-force algorithm is impractical 
for large ». We can find a fairly efficient algorithm by making some observations 
about the properties of optimal binary search trees. 

Leta) <a; < °-* <a, be the a keys to be represented in a binary search 
tree. Let 7,, denote an optimal binary search tree for a4), °**. aj, i<j. By 
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convention 7;; is an empty tree for 0 $i $n, and Tj; is not defined for i > j. Let 
Ci; be the cost of the search tree Ty. By definition ci will be 0. Let ry be the 
Toot of Ty, and let 


dj 
we=Git YL e+ Pe) 


k=i+] 


be the weight of Tj; By definition ry = 0, and wy = 4;, 0Si <n, Therefore, 
Tonis an Optimal binary search tree for a}, -*-,4,. Its cost is Coq, its weight is 
Won, and its root is ro, 

If T;; is an optimal binary search tree for a;41, ***, aj, and rj = & then k 
Satisfies the inequality i <k <j. Ty has two subtrees L and R. Lis the left sub- 
tree and contains the keys aj41, *+*,s @y-1, and R is the right subtree and contains 
the keys aj,,, ---, a; (Figure 10.5). The cost cj of Ty is 


Cij = Py + cost (L) + cost (R) + weight (L) + weight (R) (10.2) 


Where weight (L) = weight (7;,_;) = wi,-1, and weight (R) = weight (Ty) = 4y- 


Figure 10.5: An optimal binary search tree Ty 


From Eq. (10.2) it is clear that if iy is to be minimal, then cost(L) = ¢4-1 
and cost(R) = €4j, a8 otherwise we could replace either L or R by a subtree with a 


lower cost, thus getting a binary search tree for a;,,, «+, @; with a lower cost 
than ¢jj. This violates the assumption that 7;; is optimal. Hence, Eq. (10.2) 
becomes 


Cay = Pk Cy g-1 + Cag FW iad ey 
= Way + Cig + Cy (10.3) 
Since T,, is optimal, it follows from Eq. (10.3) that 73 = 4 is such that 


Wip + Cigat # Cay = mintwiz eit cy) 
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or 
Sika F C4 = Min (ci y-1 + Cy} 0.4) 
ictsj 


Equation (10.4) gives us a means of obtaining T, and i the 
knowledge that 7;; = 0 and c;, = 0. oe nscinii initia 


Example 10.3: Let n = 4 and (@;,42,43,44) = (10, 15, 20, 25). Let (p1, pa» Pa 
Pa) = (3, 3, 1, 1) and (go, 41, 92, 93. G4) = (2, 3, 1, 1, 1), The p's and q’s have 
been multiplied by 16 for convenience, Initially, 
Wu = Gir Cy =, and rj, = 0,0Si<4. Using Eqs. (10.3) and (10.4), we get 


Wor = Pit Woo + Wi =Py +41 + Wo =8 
Cor = Wor + min{cgg +c4,} =8 

ro = 1 

Wie = P2+ wi +22 =Pp2 +4241 =7 
C12 = Wy tminfey +2} =7 

re = 2 

W3 = pPatwntw3=p3+q3 tw =3 
C23 = W23 + min{cz, +033} =3 


3 = 3 

W345 Pat 33 + 4g = Py + gg + W33 = 3 
C33 = W3q + min{cs3 + Cag) = 3 

ry = 4 


Knowing Wj;4; and ¢);4;,0Si<4, we can use Eqs. (10.3) and (10.4) 
again to compute w, 542, Ci.i+2s 1,42, OS i < 3. This process may be repeated 
until wos, Coy. and rpg are obtained. The table of Figure 10.6 shows the results 
of this computation. From the table, we see that ¢o4 = 32 is the minimal cost of a 
binary search tree for a, to ay. The root of tree To4 is az. Hence, the left sub- 
uve is Ty and the right subtree 725. To, has root a; and subtrees Typ and 7),. 
T, has root a3; its left subtree is therefore 72 and right subtree 734. Thus, with 
the data in the table it is possible to reconstruct To. Figure 10.7 shows Toy. O 


Example {0.3 illustrates how Eq. (10.4) may be used to determine the c’s 
and r's, as well as how to reconstruct Tp, knowing the 7's. Let us examine the 
complexity of this function to evaluate the c’s and r’s. The evaluation function 
described in Example 10.3 requires us to compute c,, for (j —#) = 4, 2, +++, nin 
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Wo4 = 16 
4 \co4 = 32 
roy = 2 


hi... 
Figure 10.6: Computati on is Carri 
0: putation of coy and roy. The computation is carried out by row 
from row 0 to row 4 a os 2 


5) 


Figure 10.7: Optimal binary search tree for Example 10.3 
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that order. When j — i =m, there are n~m + 1 ¢jj'$ to compute. The computa- 
tion of each of these ¢;;’s requires us to find the minimum of m quantities (see 
Eq. (10.4)), Hence, each such jj can be computed in time O(n). The total time 


for all ¢);'s with j — i = mis therefore O(am - m?). The total time to evaluate all 
the c's and ryj's is 


L (um - m?) = O@) 
Isinst 


Actually we can do better than this using a result due to D, E. Knuth that 
states that the optimal / in Eq. (10.4) may be found by limiting the search to the 
range rj) SIS rj41,;- In this case, the computing time becomes O(n?) (see 
Exercise 3). Function Obst (Program 10.1) uses this result to obtain in O(?) 
time the values of w,;, rjj, and cj, 07S j<n. The actual tree To, may be con- 
structed from the values of ry in O(7t) time. The algorithm for this is left as an 
exercise. 

___ Function Obst (Program 10.2) computes the cost ¢[i]{j] = ce of optimal 
binary search trees 7; for keys a4), --*, a;. It also computes r(i]{j] = rij, the 
foot of 7,;. w[i]Lj] = wi; is the weight of Ths The two-dimensional arrays c, r 
and w are globai arrays of type int. The inputs to this function are the success 
and failure probability arrays, p{ } and q[ ] and the number of keys n. The array 
elements p[0] and a{0] are not used. 


EXERCISES 


1. (a) Prove by induction that if Tis a binary tree with n internal nodes, / its 
internal path length, and E its external path length, then E = / + 2n, 
n20. 

(b) Using the result of (a), show that the average number of comparisons 
sina successful search is related to the average number of comparis- 
ons, u, in an unsuccessful search by the formula 


s=(1+laju-ln2t 


2. Use function Obst (Program 10.1), to compute w;;, rj, and ¢j,0Si <j $4, 
for the key set (a), 43, 43, 43) = (5, 10, 15, 20), with p,; = 1/20, p2 = 14, 
V0, py = 1220, go = 14, 9g, = WO, g2 = 14, 93 = 1/20, and 
44 = 1/20. Using the r;;’s, construct the optimal binary search tree. 
3. (a) Complete function Obs by providing the code for function Knuth- 
Min. 
(b) Show that the computing time complexity of Obst is On?). 
(c) Write a C++ function BST::Construct to construct the optimal binary 


Optimal Binary Search Trees 563 


aa Obst(double «p, double *q, int n) 


for (int i = 0; i <n; i++) { 
WAL] = gli}; ell) = clit] =0; initialize ; 
wfi}{i+1] = gli] + qli+1] + pli+1]; 4 optimal trees with one node 
rUil[i+]] =i +1; 
cfi]fi+1] = whilli+t); 
wlalla] = gin]; rfn)[n) = cfalin} = 0; 
for (int m = 2; m <= n; m++) / find optimal trees with m nodes 
for (i= 0; i <= - m; i++) 


int j=i+m; 
He e ull i ee +4U} 
tk= ij); rf 
1 Knuthotin renin value k in the range (r{i){j- 1), rlé + NUN 
# minimizing c[é}[k — 1] + efk}j) 
LALA] = wll) + cfi}(k - 1) + cfk]Ui); 4 Eq. (10.3) 
nD) =k; 


} 
}/ end of Obst 


Program 10.1: Finding an optimal binary search tree 


Search tree To, given the roots ry, 0S i <j $n. Show that this can 
be done in time Ofn). 


4. Implement in C++ a BST constructor that constructs an optimal binary 
Search tree, given the success and failure probability arrays p[ ] and g[ ], 
the array of keys af }, and the number of keys n. 

Ss 


Since, often, only the approximate values of the p's and q's are known, it is 
Perhaps just as meaningful to find a binary search wee that is nearly 
Optimal (i.e., its cost, Eq. (10.1), is almost minimal for the given p's and 
q's). This exercise explores an O(n log 2) algorithm that results in nearly 
Optimal binary search trees. The search tree heuristic we shall study is 


Choose the root a, such that. Wox-i — Weal iS as small as possible. Repeat 
this process to find the left and right subtrees of a. 


(a) Using this heuristic obtain the resulting binary search tree for the 
data of Exercise 2. What is its cost? 
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(b) Write a C++ function implementing the above heuristic. The time 
complexity of your function should be O(n log 7). 


An analysis of the performance of this heuristic may be found in the paper 
by Mehlhorn (see the References and Selected Readings section). 


10.2. AVL TREES 


Dynamic collections of elements may also be maintained as binary search trees. 
In Chapter 5, we saw how insertions and deletions can be performed on binary 
search trees. Figure 10.8 shows the binary search tree obtained by entering the 
months JANUARY to DECEMBER in that order into an initially empty binary 
search tree by using function Insert (Program 5.21). 


Gu 


Figure 10.8: Binary search tree obtained for the months of the year 


The maximum number of comparisons needed to search for any key in the 
tree of Figure 10.8 is six for NOVEMBER. The average number of comparisons 
is (1 for JANUARY + 2 each for FEBRUARY and MARCH + 3 each for 
APRIL, JUNE and MAY + --- +6 for NOVEMBER)/12 = 42/12 = 3.5. If the 
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Months are entered in the order JULY, FEBRUARY, MAY, AUGUST, 
DECEMBER, MARCH, OCTOBER, APRIL, JANUARY, JUNE, SEPTEMBER, 
NOVEMBER, then the tree of Figure 10.9 is obtained. 


GH) CB) Gh GD CH 


Figure 10.9: A balanced tree for the months of the year 


The tree of Figure 10.9 is well balanced and does not have any paths toa 
leaf node that are much longer than others. This is nog true of the tree of Figure 
10.8, which has six nodes on the path from the root to NOVEMBER and only 
two nodes on the path to APRIL, Moreover, during the.construction of the tree 
of Figure 10.9, all intermediate trees obtained are also well balanced. The max- 
imum number of key comparisons needed to find any key is now 4, and the aver- 
age is 37/12 = 3.1. If the months are entered in lexicographic order, instead, the 
tree degenerates to a chain as in Figure 10.10. The maximum search time is now 
12 key comparisons, and the average is 6.5.. Thus, in the worst case, searching a 
binary search tree corresponds to sequential searching in a sorted linear list. 
When the keys are entered in a random order, the ree tends to be balanced-as_in 
Figure_10.9. If all Pgrmutations are equally probable, then the average search 
and insertion time is O(log 7) for an n-node binary search tree. 

From our earlier study of binary tees, we know that both the average and 
maximum search time will be minimized if the binary search tree is maintained 
as a complete binary tree at all times. However, since we are dealing with a 
dynamic situation, it is difficult to achieve this ideal without making the time 
required to insert a key very high. This is so because in some cases it would be 
necessary to restructure the whole tee to accommodate the new entry and at the 
Same time have a complete binary search tree. It is, however, possible to keep 
the tree balanced to ensure both an average and worst-case search time of 
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Figure 10.10: Degenerate binary search tree 


O(log 7) for a tree with nodes. In this section, we study one method of grow- 
ing balanced binary trees. These balanced trees will have satisfactory search, 
insertion and deletion time properties. Other ways to maintain balanced search 
trees are studied in later sections. 

In 1962, Adelson-Velskii and Landis introduced a binary tree structure that 
is balanced with respect to the heights of subtrees. As a result of the balanced 
nature of this type of tree, dynamic retrievals can be performed in O(og 7) time 
if the tree has » nodes in it. At the same time, a new key can be entered or 
deleted from such a tree in time O(log). The resulting tree remains height- 
balanced. This tree structure is called an AVL tree. As with binary trees, it is 
natural to define AVL trees recursively. 
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Definition: An empty tree is height-balanced. If T is a nonempty binary uee 
with T, and Tp as its left and right subtrees respectively, then T is height- 
balanced iff (1) T, and Tg are height-balanced and (2) | hy ~hp | $1 where hy 
and hg are the heights of T, and Tp, respectively. 0 


The definition of a height-balanced binary tree requires that every subtree 
also be height-balanced. The binary tree of Figure 10.8 is not height-balanced, 
Since the height of the left subtree of the tree with root APRIL is 0 and that of the 
right subtree is 2. The tree of Figure 10.9 is height-balanced while that of Figure 
10.10 is not. To illustrate the processes involved in maintaining a height- 
balanced binary search tree, let us try to construct such a tree for the months of 
the year. This time let us assume that the insertions are made in the following 
order: MARCH, MAY, NOVEMBER, AUGUST, APRIL, JANUARY, 
DECEMBER, JULY, FEBRUARY, JUNE, OCTOBER, SEPTEMBER. Figure 
10.11 shows the tree as it grows and the restructuring involved in keeping the 
tree balanced. The numbers above each node represent the difference in heights 
between the left and tight subtrees of that node. This number is referred to as the 
balance factor of the node. 


Definition: The balance factor, BF (T), of a node Tin a binary tree is defined to 
be hy ~ hp, where hy and hp, respectively, are the heights of the Jeft and right 
subtrees of T. For any node Tin an AVL wee, BF(T) = -1,0,0rJ. O 


____ Inserting MARCH and MAY results in the binary search trees (a) and (b) of 
Figore 10.11. When NOVEMBER is inserted into the tree, the height of the right 
subtree of MARCH becomes 2, whereas that of the left subtree is 0. The tree has 
become unbalanced. To rebalance the tree, a rotation is performed. MARCH is 
made the left child of MAY, and MAY becomes the root (Figure 10.11(c)). The 
introduction of AUGUST leaves the tree balanced (Figure 10.11(d)). : 

The next insertion, APRIL, causes the tree to become unbalanced again. 
To rebalance the tree, another rotation is performed. This time, it is a clockwise 
rotation. MARCH is made the right child of AUGUST, and AUGUST becomes 
the root of the subtree (Figure 10.1 1(e)). Note that both of the previous rotations 
Were carried out with respect to the closest parent of the new node that had a bal- 
ance factor of +2. The insertion of JANUARY results in an unbalanced tree. 
This time, however, the rotation involved is somewhat more complex than in the 
earlier situations. The common point, however, is that the rotation is still carried 
Out with Tespect to the nearest parent of JANUARY with a balance factor £2. 
MARCH becomes the new root. AUGUST, together with its left subuee, 
becomes the left subuee of MARCH. The left subtree of MARCH becomes the 

Tight subtree of AUGUST. MAY and its right subtree, which have keys greater 

than MARCH, become the right subtree of MARCH. (If MARCH had had a 
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Figure 10.11: Balanced trees obtained for the months of the year (continued on 
next page) 
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(g) Insert DECEMBER (h) Insert JULY 


Figure 10,11: Balanced trees obtained for the months of the year (continued on 
next page) 


nonempty right subtree, this could have become the left subtree of MAY, since 
all keys would have been less than MAY.) 

Inserting DECEMBER and JULY necessitates no rebalancing. When 
FEBRUARY is inserted, the tree becomes unbalanced again. The rebalancing 
process is very similar to that used when JANUARY was inserted. The nearest 
parent with balance factor +2 is AUGUST. DECEMBER becomes the new root 
of that subtree. AUGUST, with its left subtree, becomes the left subtree. JANU- 
ARY, with its right subtree, becomes the right subtree of DECEMBER; FEBRU- 
ARY becomes the left subtree of JANUARY. (If DECEMBER had had a left 
subtree, it would have become the right subtree of AUGUST.) The insertion of 
JUNE requires the same rebalancing as in Figure 10.11(f). The rebalancing 
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(j) Insert JUNE 


Figure 10.11; Balanced trees obtained for the months of the year (continued on 
next page) 


following the insertion of OCTOBER is identical to that following the insertion 
of NOVEMBER. Inserting SEPTEMBER leaves the tree balanced. 

____ Inthe preceding example we saw that the addition of a node to a balanced 
binary search tree could unbalance it. The rebalancing was carried out using 
four different kinds of rotations: LL, RR, LR, and RL (Figure 10.11 (e). (0) (. 
and (i), respectively). LL and RR are symmetric, as are LR and RL- These 
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(1) Insert SEPTEMBER 


Figure 10.11: Balanced trees obtained for the months of the year 


rotations are characterized by the nearest ancestor, A, of the inserted node, Y, 
whose balance factor becomes £2. The following characterization of rotation 
types is obtained: 

LL: new node Y is inserted in the left subtree of the left subtree of A 

LR: is inserted in the right subtree of the lett subtree of A 

RR: Y is inserted in the right subtree of the right subtree of A 
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RL: Y is inserted in the left subtree of the right subtree of A 


A moment’s reflection will show that if a height-balanced binary tree 
becomes unbalanced as a result of an insertion, then these are the only four cases 
possible for rebalancing. Figures 10.12 and 10.13 show the LL and LR rotations 
in terms of abstract binary trees. The RR and RL rotations are symmetric. The 
root node in each of the trees of the figures represents the nearest ancestor whose 
balance factor has become +2 as a result of the insertion. In the example of Fig- 
ure 10.11 and in the rotations of Figures 10.12 and 10.13, notice that the height 
of the subtree involved in the rotation is the same after rebalancing as it was 
before the insertion. This means that once the rebalancing has been carried out 
on the subtree in question, examining the remaining tree is unneccessary. The 
only nodes whose balance factors can change are those in the subtree that is 
Totated. 


A 
% 
B Ae 
h 
By Br {Be Br Br AR 
h h UAL h . A A 
(a) Before insertion (b) After inserting imo, (c) After LL rotation 


Balance factors are inside nodes 
Subtree heights are below subtree names 


Figure 10,12: An LL rotation 


The transformations done to remedy LL and RR imbalances are often 
called single rotations, while those done for LR and RL imbalances are called 
double rotations. The transformation for an LR imbalance can be viewed as an 
RR rotation followed by an LL rotation, while that for an RL imbalance can be 
viewed as an LL rotation followed by an RR rotation. 

To carry out the rotations of Figures 10.12 and 10.13, it is necessary to 
Jocate the node A around which the rotation is to be performed. As remarked 
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(a) Before insertion (b) After inserting into By (c) After LR rotation 


b=0=06f (B)=bf (A)=0 after rotation 
b=1=9bf (B)=0 and bf (A)=-1 after rotation 
b=—l=bf (B)=1 and bf (A)=0 after rotation 


Figure 10.13: An LR rotation 


earlier, this is the nearest ancestor of the newly inserted node whose balance fac- 
tor becomes +2. For a node’s balance factor to become +2, its balance factor 
must have been +] before the insertion. Therefore, before the insertion, the bal- 
ance factors of all nodes on the path from A to the new insertion point must have 
been 0. With this information, the node A is readily determined to be the nearest 
ancestor of the new node having a balance factor +1 before insertion. To com- 
plete the rotations, the address of F, the parent of A, is also needed. The changes 
in the balance factors of the relevant nodes are shown in Figures 10.12 and 
10.13. Knowing F and A, these changes can be carried out easily. 

What happens when the insertion of a node does not result in an unbal- 
anced tree (see Figure 10.11 (a), (b), (4), (g), (h), and (1)? Although no restruc- 
turing of the tree is needed, the balance factors of several nodes change. Let A 
be the nearest ancestor of the new node with balance factor +1 before insertion. 
If, as a result of the insertion, the tree did not get unbalanced, even though some 
path length increased by 1, it must be that the new balance factor of A is 0. If 
there is no ancestor A with a balance factor +] (as in Figure 10.11 (a), {b), (d), 
(g), and (1)), let A be the root. The balance factors of nodes from A to the parent 
of the new node will change to +] (see Figure 10.11 (h); A= JANUARY). Note 
that in both cases, the procedure to determine A is the same as when rebalancing 
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is needed. The remaining details of the insertion-rebalancing process are speiled 
out in function Av/:-dnsert (Program 10.3). The class definitions in use are 


template <class K, class E> class AVL; 


template <class K, class E> 
class AviNode { 
friend class AVL<K, E>; 
public AviNode(const K& k, const E& e) 
{key = k; element = e; bf=0; leftChild = rightChild = 03} 


private: 
K key; 
E element; 
int bf; 
AviNode<K, E> *leftChild, trightChild; 
k 
template <class K, class E> 
class AVL { 
public: 
AVL() : root (0) (}3 
E& Search(const K&) const; 4 
void Insert(const K&, const E&); 
void Delete(const K&); 
private: 
AviNode<K, E> *root; 
k 


Here, E is the data type of the elements in the AVL tree and K is the data 
type of the keys. 


template <class K, class E> 
void AVL<K, E>::insert(const K& k, const E& e) 
( 
if (‘root) { # special case: empty tree 
root = new AviNode<K, E> (k, €); 
return; 
} 
# Phase !: Locate insertion point for e. 
AvlNode<K, E> *a=0, // most recent node with bf = +1 
*pa =0, 4 parent of a 
*p = root, 4 p moves through the tree 
*pp =0; # parent of p 
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while(p) { 
if (pf) ta = ps pa = pps} 
if (k < pkey) (pp = ps p = pleftChilds} 11 take left branch 
else if (k > pkey) {pp = p3 p = prightChild;} 
else {p—> element = e; return;} //k already in tree 


} 4 end of while 


Phase 2: Insert and rebalance. & is not in the tree and 

/f may be inserted as the appropriate child of pp. 
AviNode<K, E> +y = new AviNode<T>{k, €); 

if (k < pp key) pp—rleftChild = y; H insert as left child 
else pp—rightChild = y; 4 insen as right child 


//Adjust balance factors of nodes on path from a to pp. By the definition 
// of a, all nodes on this path presently have a balance factor of 0. Their new 
/# balance factor will be t1. d = +1 implies that k is inserted in the left subtree 
“4 of a. d =—1 implies that & is inserted in the right subtree of a. 
int d; 
AviNode<K, E> *b, Hf child of a 

4c; I child of b 
if (k > a—key) (b = p = a-srightChild; d= —-1;} ) 
else {b a—sleftChild; d= 13} 
while (p != y) 

if (k > pkey) { / height of right increases by | 
pbf =-1; p = prightChild; 


} 

else { // height of left increases by } 
pbf = 1; p= pleftChild; 

} 


Is tree unbalanced? 
if ( {(a—9bf) Il (a—sbf + d)) { wee still balanced 
a bf += d; return; 


# tree unbalanced, determine rotation type 
if (d == 1) {/ left imbalance 
if (b-bf == 1) {# rotation type LL 
a-lefiChild = b-rightChild; 
brightChild = a; a>bf = 0; b>bf = 0; 


} 
else { // rotation type LR 
¢ = brightChild; 
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brightChild = cleftChild; 
a-lefiChild = crightChild; 
coleftChild = b; 
eorightChild = a; 
switch (cbf) { 
case J: 
a bf =-1; b> bf=0; 
break; 
case~]: 
b—-bf = 1; abf=0; 
break; 
case 0: 
b-bf = 0; abf = 0; 
break; 


} 
cbf = 0; b = c; 1 bis the new root 
}4end of LR 
} # end of left imbalance 
else { // right imbalance: this is symmetric to left imbalance 


) 


Hf Subtree with root b has been rebalanced. 
if (t1pa) root = b; 
else if (a == pa—leftChild) pa~leftChild = b; 
else parightChild = b; 
return; 
} Mend of AVL::Insert 


Program 10.3: Insertion into an AVL tree 


To really understand the insertion algorithm, you should try it out on the 
example of Figure 10.11. An analysis of the algorithm reveals that if h is the 
height of the tree before insertion, then the time to insert a new key is Oh). This 
is the same as for unbalanced binary search trees, although the overhead Is 
significantly greater now. In the case of binary search trees, however, if cat 
were n nodes in the tree, then A could, in the worst case, be # (Figure 10.10), an 
the worst case insertion time would be O(n). In the case of AVL trees, ak 
h can be at most O(log n), so the worst-case insertion time is O(log n). Let 
this, let Nj, be the minimum number of nodes in a height-balanced tree of ae 
A. In the worst case, the height of one of the subtrees will be  ~ 1 and tl oe 
the other h-2. Both of these subtrees are also height balanced. ee 
Ni = Nyy +Np2 +1, and Ny = 0, N, =1 and Nz = 2. Note the simu 
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between this recursive definition for N, and that for the Fibonacci nuinbers 
F, = Fa-t + Fu-2» Fo = 0, and F, = 1. In fact, we can show that Nj, = Fh gz ~ 1 
for A 20 (see Exercise 3). From Fibonacci number theory it is known that 
Fy, = 98/15, where o = (1 + YSV2. Hence, N, = ¢"*? /¥5—1. This means that 
if there are n nodes in the tree, then its height, A, is at most logy (VS(n + 1)) -2- 
1.44log(n + 1)—0.33. The worst-case insertion time for a height-balanced tree 
with n nodes is, therefore, O(log n). 

The exercises show that it is possible to find and delete a node with key X 
and to find and delete the kth node from a height-balanced tee in O(log n) time. 
Results of an empirical study of deletion in height-balanced wees may be found 
jn the paper by Karlton et al. (see the References and Selected Readings section). 
Their study indicates that a random insertion requires no rebalancing, a rebalanc- 
ing rotation of type LL or RR, and a rebalancing rotation of type LR and RL, 
with probabilities 0.5349, 0.2327, and 0.2324, respectively. Figure 10.14 com- 
pares the worst-case times of certain operations on sorted sequential lists, sorted 
linked lists, and AVL trees. 


Operation 
‘Search for element with key k 
Search for jth item 

Delete element with key k 
Delete jth element 

Insert 

Output in order 


Linked list, AVL wee | 
O(log n) O(n) O(log n) 
OU) OU) Ovlog n) 
O(n) oc)! O(log 2) | 
O(n -j) Oy) O(log n) | 
O(n) ou? O(log») | 
Of) O(n) O(n) } 
1. Doubly linked list and position of k known 

2. Position for insertion known 


Figure 10.14: Comparison of various structures 


EXERCISES 


1, (a) Convince yourself that Figures 10.12 and 10.13 together with the 
cases for the symmetric rotations RR and RL takes care of all the 
possible situations that may arise when a height-balanced binary tree 
becomes unbalanced as a result of an insertion. Alternately, come up 
with an example that is not covered by any of the cases in this figure. 


(b) Draw the transformations for the rotation types RR and RL. 
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Me 
12. 


Show that the LR rotation of Figure 10.13 is equivalent to an RR tolatioy 
followed by an LL rotation and that an RL rotation is equivalent to an LL 
rotation followed by an RR rotation. 


Prove by induction that the minimum number of nodes in an AVL tree of 
height # is Ny, = F,42- 1,4 20. 

Complete Avi::Insert (Program 10.3) by filling in the code needed to rebal- 
ance the tree in case of a right imbalance. 


Start with an empty AVL tree and perform the following sequence of insey- 
tions: DECEMBER, JANUARY, APRIL, MARCH, JULY, AUGUST. 
OCTOBER, FEBRUARY, NOVEMBER, MAY, JUNE. Use the strategy of 
Avl:Insert to perform each insert. Draw the AVL tree following each inser- 
tion and state the rotation type (if any) for each insert. 


Assume that each node in an AVL tree has the data member Isize. For any 
node, a, a—slsize is the number of nodes in its left subtree plus one. Write 
a C++ function Avi::Find{k) to locate the kth smallest key in the tree, 
Show that this can be done in O(log 7) time if there are n nodes in the tree. 
Rewrite Avi::/nsert with the added assumption that each node has an lsize 
data member as in Exercise 6. Show that the insertion time remains 
O(log 2). 

Write a C++ function to list the elements of an AVL tree in ascending order 
of key. Show that this can be done in O(t) time if the tree has n nodes. 
Write an algorithm to delete the element with key & from an AVL tree. The 
resulting tree should be restructured if necessary. Show that the time 
required for this is O(log n) when there are n nodes in the tree. [Hint: If k is 
not in a leaf, then replace & by the largest value in its left subtree or the 
smallest value in its right subtree. Continue until the deletion propagates 
to a leaf. Deletion from a leaf can be handled using the reverse of the 
twansformations used for insertion.] 

Do Exercise 9 for the case when each node has an Isize data member and 
the kth smallest key is to be deleted. 


Complete Figure 10.14 by adding a column for hashing. 

For a fixed k, k 2 1, we define a height-balanced tree HB (k) as below: 
Definition: An empty binary tree is an HB(k) ree. If Tis a money 
binary tree with 7; and Tp as its left and right subtrees, then Tis He 
(a) T, and Tg are HB(k) and (b) | hy — gl Sk, where hy and Ag are the 
heights of 7, and Tp, respectively. 0 


(a) Obtain the rebalancing transformations for HB (2). 
(b) Write an insertion algorithm for HB (2) trees. 
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10.3 RED-BLACK TREES 
10.3.1 Definition 


A red-black tree is a binary search tree in which every node is colored either red 
or black. The remaining properties satisfied by a red-black tree are best stated in 
terms of the corresponding extended binary tree. Recall, from Section 9.2, that 
we obtain an extended binary tree from a regular binary tree by replacing every 
null pointer with an external node. The additional properties are 


RB1. = The root and all external nodes are colored black. 
RB2. No root-to-external-node path has two consecutive red nodes. 
RB3. All root-to-external-node paths have the same number of black nodes. 


An equivalent definition arises from assigning colors to the pointers 
between a node and its children. The pointer from a parent to a black child is 
black and to a red child is red. Additionally, 


RB1% Pointers from an internal node to an extemal node are black. 
RB2* No root-to-external-node path has two consecutive red pointers. 
RB3% All root-to-extemal-node paths have the same number of black pointers. 


Notice that if we know the pointer colors, we can deduce the node colors 
and vice versa. In the red-black wee of Figure 10.15, the external nodes are 
shaded squares, black nodes are shaded circles, red nodes are unshaded circles, 
black pointers are thick lines, and red pointers are thin lines. Notice that every 
path from the root to an external node has exactly two black pointers and three 
black nodes (including the root and the external node); no such path has two con- 
secutive red nodes or pointers. 

Let the rank of a node in a red-black tree be the number of black pointers 
(equivalently the number of black nodes minus 1) on any path from the node to 
any external node in its subtree. So the rank of an external node is 0. The rank 
of the root of Figure 10.15 is 2, that of its left child is 2, and of its right child is 1. 


Lemma 10.1: Let the length of a root-to-external-node path be the number of 
pointers on the path. If P and Q are two root-to-external-node paths in a red- 
black tree, then length (P) < 2length (Q). 


Proof: Consider any red-black tree. Suppose that the rank of the root is r. From 
RBI ‘the last pointer on each root-to-extemnal-node path is black. From RB2 ‘no 
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ras 


Figure 10.15: A red-black tree 


(a 


such path has two consecutive red pointers. So each red pointer is followed by a 
black pointer. As a result, each root-to-external-node path has between r and 2r 
pointers, so lengrh(P) < 2lengrh(Q). To see that the upper bound is possible, 
consider the red-black tree of Figure 10.15. The path from the root to the left 
child of 5 has length 4, while that to the right child of 80 has length 2. 0. 


Lemma 10.2: Let h be the height of a red-black tree (excluding the extemal 
nodes), let # be the number of internal nodes in the tree, and let r be the rank of 
the root. 


(a) A<2r 
(b) 1227-1 
(c) AS 2logo(n th) 


Proof: From the proof of Lemma 10.1, we know that no root-to-external-node 
path has length > 2r, so ft $ 2r. (The height of the red-black tree of Figure 10.15 
with external nodes removed is 2r = 4.) 

Since the rank of the root is r, there are no external nodes at levels 1 
through r, so there are 2-1 internal nodes at these levels. Consequently, the 
total number of internal nodes is at least this much. (In the red-black tree of Fig- 
ure 10.15, levels 1 and 2 have 3 = 22-1 intemal nodes. There are additional 
internat nodes at levels 3 and 4.) 7 

From (b) it follows that r < log;(n+1). This inequality together with (a) 
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yields (c). B 


Since the height of a red-black tree is at most 2log,(n+1), search, insert, 
and delete algorithms that work in OA) time have complexity O(log n). 


Notice that the worst-case height of a red-black tree is more than the worst-case 
height (approximately 1.44log,(n +2)) of an AVL tree with the same number of 
(internal) nodes. 


10.3.2 Representation of a Red-Black Tree 


Although it is convenient to include external nodes when defining red-black 
trees, in an implementation null pointers, rather than physical nodes, represent 
external nodes. Further, since pointer and node colors are closely related, with 
each node we need to store only its color or the color of the two pointers to its 
children. Node colors require just one additional bit per node, while pointer 
colors require two. Since both schemes require almost the same amount of 
space, we may choose between them on the basis of actual run times of the 
resulting red-black tree algorithms. 

In our discussion of the insert and delete operations, we will explicitly state 
the needed color changes only for the nodes. The corresponding pointer color 
changes may be inferred. 


10.3.3. Searching a Red-Black Tree 


‘We can search a red-black tree with the code we used to search an ordinary 
binary search tree (Program 5.19). This code has complexity O(4), which is 
O(log )} for a red-black tree. Since we use the same code to search ordinary 
binary search trees, AVL trees, and red-black trees and since the worst-case 
height of an AVL tree is least, we expect AVL trees to show the best worst-case 
performance in applications where search is the dominant operation. 


10.3.4 Inserting into a Red-Black Tree 


Elements may be inserted using the strategy used for ordinary binary trees (Pro- 
gram 5.21). When the new node is attached to the red-black tree, we need to 
assign the node a color. If the tree was empty before the insertion, then the new 
node is the root and must be colored black (see property RB1). Suppose the tree 
was not empty prior to the insertion. If the new node is given the color black, 
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then we will have an extra black node on paths from the root to the extemal 
nodes that are children of the new node. On the other hand, if the new node is 
assigned the color red, then we might have two consecutive red nodes, Making 
the new node black is guaranteed to cause a violation of property RB3, while 
making the new node red may or may not violate property RB2. We will make 
the new node red. 

If making the new node red causes a violation of property RB2, we will say 
that the tree has become imbalanced. The nature of the imbalance is classified 
by examining the new node x, its parent pu, and the grandparent gu of u. 
Observe that since property RB2 has been violated, we have two consecutive red 
nodes, One of these red nodes is u, and the other must be its parent; therefore, pu 
exists. Since pu is red, it cannot be the root (as the root is black by property 
RBI); u must have a grandparent gu, which must be black (property RB2), 
When pu is the left child of gu, « is the left child of pu and the other child of gu 
is black (this case includes the case when the other child of gu is an extemal 
node); the imbalance is of type LLb. The other imbalance types are LLr (pu is 
the left child of gu, u is the left child of pu, the other child of gu is red), LRb (pu 
is the left child of gu, u is the right child of pu, the other child of gu is black), 
LRr, RRb, RRr, RLb, and RLr. 

Imbalances of the type XYr (X and Y may be L or R) are handled by 
changing colors, while those of type XYb require a rotation. When we change a 
color, the RB2 violation may propagate two levels up the tree. In this case we 
will need to reclassify at the new level, with the new u being the former gu, and 
apply the transformations again. When a rotation is done, the RB2 violation is 
taken care of and no further work is needed. 

Figure 10.16 shows the color changes performed for LLr and LRr imbal- 
ances; these color changes are identical. Black nodes are shaded, while red ones 
are not. In Figure 10.16(a), for example, gu is black, while pu and u are red; the 
pointers from gu to its left and right children are red; gitp is the right subtree of 
gu; and pug is the right subtree of pu. Both LLr and LRr color changes require 
us to change the color of pu and of the right child of gu from ted to black. Addi- 
tionally, we change the color of gu from black to red provided gu is not the root. 
Since this color change is not done when gu is the root, the number of black 
nodes on all root-to-external-node paths increases by | when gu is the root of the 
red-black tree. 

If changing the color of gu to red causes an imbalance, gu becomes the mal 
u node, its parent becomes the new pu, its grandparent becomes the new gu, am 
we continue to rebalance. If gu is the root or if the color change does not cause 
an RB2 violation at gu, we are done. , 

Figure 10.17 oils the rotations performed to handle LLb and LRb imbal- 
ances. In Figures 10.17(a) and (b), # is the root of pu,. Notice the similarity 
between these rotations and the LL (refer to Figure 10.12) and LR (refer to 
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(4) After LRr color change 


Figure 10.16; LLr and LRr color changes 


Figure 10.13) rotations used to handle an imbalance following an insertion in an 
AVL tree. The pointer changes are the same. In the case of an LLb rotation, for 
example, in addition to pointer changes we need to change the color of gu from 


black to red and of pu from red to black. 


In examining the node (or pointer) colors after the rotations of Figure 
10.17, we see that the number of black nodes (or pointers) on all root-to- 
external-node paths is unchanged. Further, the root of the involved subtree (gut 
before the rotation and pu after) is black following the rotation; therefore, two 
consecutive red nodes cannot exist on the path from the tree root to the new pu. 
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gu pu 
pu bur pur gu 
Pu Pug puR Bug 
(a) LLb imbalance (b) After LLb rotation 
gu 
pu 84ty u 
Pu, ou pu gu 
uy up Puy up uR bur 
(c) LRb imbalance (d) After LRb rotation 


Figure 10.17: LLb and LRb rotations for red-black insertion 


Consequently, no additional rebalancing work is to be done. A single rotation 
(preceded possibly by O(log n) color changes) suffices to restore balance follow- 
ing an insertion! 


Example 10.4: Consider the red-black tree of Figure 10.18(a). External nodes 
are shown for convenience. In an actual implementation, the shown black 
pointers to external nodes are simply null pointers and external nodes are not 
represented. Notice that ail root-to-extenal-node paths have three black nodes 
(including the externa] node) and two black pointers. 

To insert 70 into this red-black tree, we use the algorithm of Program 10.4, 
The new node is added as the left child of 80. Since the insertion is done into a 
nonempty tree, the new node is assigned the color red. So the pointer to it from 
its parent (80) is also red. This insertion does not result in a violation of property 
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my J 


(a) Initial (b) Insert 70 


(c) Insert 60 (d) LLr color change 


(e) Insert 65 (f) LRb rotation 


Figure 10.18: Insertion into a red-black tree (continued on next page) 
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RB2, and no remedial action is necessary. Notice that the number of black 
pointers on all root-to-extemal-node paths is the same as before the insertion, 

Next insert 60 into the tree of Figure 10.18(b). The algorithm of Program 
5.23 attaches a new node as the left child of 70, as is shown in Figure 10.18(c), 
The new node is red, and the pointer to it is also red. The new node is the a 
node, its parent (70) is pu, and its grandparent (80) is gu. Since pu and u are red 
we have an imbalance. This imbalance is classified as an LLr imbalance (as mu 
is the left child of gu, u is the left child of pu, and the other child of gu is red), 
When the LLr color change of Figure 10.16(a) and (b) is performed, we get the 
tree of Figure 10.18(d). Now «, pu, and gu are each moved two levels up the 
tree. The node with 80 is the new « node, the root becomes pu, and gu is NULL, 
Since there is no gu node, we cannot have an RB2 imbalance at this location and 
we are done. All root-to-external-node paths have exactly two black pointers, 

Now insert 65 into the tree of Figure 10.18(d). The result appears in Figure 
10.18(e). The new node is the u node. Its parent and grandparent are, respec- 
tively, the pu and gu nodes. We have an LRb imbalance that requires us to per- 
form the rotation of Figures 10.17(c) and (d). The result is the tree of Figure 
10.18(f). 

Finally, insert 62 to obtain the tree of Figure 10.18(g). We have an LRr 
imbalance that requires a color change. The resulting tree and the new wu, pu, and 
gu nodes appear in Figure 10.18(h). The color change just performed has caused 
an RLb imbalance two levels up, so we now need to perform an RLb rotation. 
The rotation results in the tree of Figure 10.18(i). Following a rotation, no 
further work is needed, and we are done. 0 


10.3.5 Deletion from a Red-Black Tree 
The development of the deletion transformations is left as an exercise. 


10.3.6 Joining Red-Black Trees 


In Section 5.7.5, we defined the following operations on binary search trees: 

ThreeWayJoin, TwoWayJoin, and Split. Each of these can be performed in loga- 

rithmic time on red-black trees. The operation C.ThreeWayJoin (A, x, B) can be 

performed as follows. A two-way join may be done in a similar fashion. 

Case 1: If A and B have the same rank, then let C be constructed by creating a 
new root with pair x, leftChild A, and rightChild B. Both links are 
made black. The rank of C is one more than the ranks of A and B. 

Case 2: If rank(A) > rank(B), then follow rightChild pointers from A to the 
first node ¥ that has rank equal to rank(B). Properties RBI to RB3 
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(g) Insert 62 (h) LRr color change 


{i) RLb rotation 


Figure 10.18: Insertion into a red-black tree 


guarantee the existence of such a node. Let p(¥) be the parent of ¥. 
From the definition of Y, it follows that rank(p(Y)) = rank(Y) + 1. 
Hence, the pointer from p(¥) to Y is a black pointer. Create a new 
node, Z, with pair x, leftChild Y (i.c., node Y and its subtrees become 
the left subtree of Z) and rightChild B. Z is made the right child of 
p(Y), and the link from p(¥) to Zhas color red. The links from Z to its 
children are made black. Note that this transformation does not change 
the number of black pointers on any root-to-extemal-node path. How- 
ever, it may cause the path from the root to Z to contain two consecu- 
tive red pointers. If this happens, then the transformations used to 
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handle this in a bottom-up insertion are performed. These transform, 
tions may increase the rank of the tree by one. % 


Case 3: The case rank (A) < rank (B) is similar to Case 2. 


Analysis of ThreeWayJoin: The correctness of the function just described is 
easily established. Case 1 takes O(1) time; each of the remaining two cases 
takes O(\rank(A)— rank (B) |) time under the assumption that the rank of each 
ted-black tree is known prior to computing the join. Hence, a three-way join can 
be done in O(log n) time, where n is the number of nodes in the two trees being 
joined. A two-way join can be performed in a similar manner. Note that there is 
no need to add parent data members to the nodes to perform a join, as the needed 
parents can be saved on a stack as we move from the root to the node ¥. 0 


10.3.7 Splitting a Red-Black Tree 


‘We now turn our attention to the split operation. Assume for simplicity that the 
splitting key, i, is actually present in the red-black tree A. Under this assump- 
tion. the split operation A . Split (i, B, x, C) can be performed as in Program 10.5. 


Step 1: Search A for the node P that contains the element with key i. Copy this 
element to the reference parameter x. Initialize B and C to be the left 
and right subtrees of P, respectively. 

Step 2: 
for (Q = parent(P); Q; P = Q, Q = parent(Q)) 

if (P == Q-leftChild) 
C.ThreeWayJoin(C, Q>data, Q—rightChild) 
else B.ThreeWayJoin(Q—leftChild, Q—data, B); 
} 


Program 10.5: Splitting a red-black tree 


We first locate the splitting element, x, in the red-black tree. Let P be the 
node that contains this element. The left subtree of P contains elements with key 
less than i. B is initialized to be this subtree. All elements in the right subtree of 
P have a key larger than i, and C is initialized to be this subtree. In Step 2, we 
trace the path from P to the root of the red-black tree A. During this traceback, 
two kinds of subtrees are encountered. One of these contains elements with keys 
that are larger than / as well as all keys in C. This happens when the traceback 
moves from a left child of a node to its parent. The other kind of subtree 
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contains elements with keys that are smatler than i, as well as smaller than ali 
keys in B. This happens when we move from a right child to its parent. In the 
former case a three-way join with C is performed, and in the latter a three-way 
join with B is performed. One may verify that the two-step procedure outlined 
here does indeed implement the split operation for the case when j is the key of 
an element in the tree A. It is easily extended to handle the case when the tree A 
contains no element with key i. 


Analysis of split: Call a node red if the pointer to it from its parent is red. The 
root and all nodes that have a black pointer from their parent are black, Let r(X) 
be the rank of node X in the unsplit tree. First, we shal) show that during a split, 
if P is a black node in the unsplit wee, and Q #0, then 
r(Q)2 max{r(B), r(C)) 

where P, Q, B, and C are as defined at the start of an iteration of the fer loop in 
Step 2. 
From the definition of rank, the inequality holds at the start of the first itera- 
tion of the for loop regardless of the color of P. If P is red initially, then its 
parent, Q, exists and is black. Let q” be the parent of Q. If q° = 0, then there is 
no Q at which the inequality is violated. So, assume q° #0. From the definition 
of rank and the fact that Q is black, it follows that r(g) = r(Q) + 1. Let B’ and 
C’ be the trees B and C following the three-way join of Step 2. Since r(B’) < 
r(B) + 1 and r(C) s r(C) +1, r(q) = r(Q) +1 2 max{r(B), r(C)) + 1 2 
max(r(B‘), r(C’)}. So, the inequality holds the first time Q points to a node 
with a black child P (i.e., at the start of the second iteration of the for loop, when 
Q=q°). 
Having established the induction base, we can proceed to show that the 
inequality holds at all subsequent iterations when Q points to a node with a black 
child P. Suppose Q is currently pointing to a node g with a black child P = p. 
Assume that the inequality holds. We shall show that it will hold again the next 
time Q is at a node with a black P. For there to be such a next time, the parent q” 
of q must exist. If g is black, the proof is similar to that provided for a black @ 
and a red P in the induction base. 

If q is red, then q’ is black. Further, for there to be a next time when Qisat 
a node with a black P, q must have a grandparent q“’, as when Q moves to q’ and 
Ptoq, Q=q’ has ared child P=q. Let B“ and C’ represent the B and C tees 
following the iteration that begins with P = p and Q = g. Similarly, tet B~” and 
C~ represent these trees following the iteration that begins with P = q and Q = 
‘ Suppose that C is joined with q and its right subtree R to create C’. If r(C) 
= r(R), then r(C’) = r(C) + 1, and C” has two black children (recall that when 
the rank increases by one, the root has two black children). If C’’ = C’ then B= 
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B’ is combined with ‘and its left subtree L’ to form B’’. Since r 
r(B’) S$ max{r(B), r(L)} + 1, and r(g”) = rq) +1 = rg)+ 1, r(B”) < 
rq"). Also, (C“) = r(C) = r(C) + LS r(q) + 1 r(q"). So, the inequalir 
holds when Q = q”. If C’’# C’, then C” is combined with q’ and its right soe 
tree R’ to form C’. If r(R’) 2 r(C’), then r(C) < r(RY +18 rq)+1= 
r(q“). If r(R) < r(C*), then r(C’’) = (C7), as C’ has two black children, and 
the join of C’, g’, and R’ does not increase the rank. Once again, res 
r(q‘’), and the inequality holds when Q = q“’. 

If r(C) > r(R) and r(C’)=r(C), then r(q“) = r(q) + 1 2 max{r(B), r(C)) 
+12 max{r(B’), r(C)}. If r(C’) = r(C) + 1, then C” has two black children, 
and r(C“’) $ r(q) + 1 = r(q”). Also, r(B“) S$ r(q‘). So, the inequality holds 
when Q = q“’. The case r(C) < r(R) is similar. 

The case when B is joined with q and its left subtree L is symmetric. 

Using the rank inequality just established, we can show that whenever Q 
points to a node with a black child, the total work done in Step 2 of the splitting 
algorithm from initiation to the time Q reaches this node is 
O(r(B) + r(C) + r(Q)). Here, B and C are, respectively, the current red-black 
wees with values smaller and larger than the splitting value. Since r(Q) 2 
max{7r(B), r(C)}, the total work done in Step 2 is O(r(Q)). From this, it follows 
that the time required to perform a split is O(log x), where n is the number of 
nodes in the tree to be split. 0 


(L$ rq" 


EXERCISES 


1, Start with an empty red-black tree and insert the following keys in the 
given order: 15, 14, 13, 12, 11, 10, 9, 8, 7, 6, 5, 4, 3, 2, 1. Draw figures 
similar to Figure 10.18 depicting your tree immediately after each insertion 
and following the rebalancing rotation or color change (if any). Label all 
nodes with their color and identify the rotation type (if any) that is done. 

2. Do Exercise | using the insert key sequence: 1, 2, 3, 4, 5,6, 7, 8.9 10, 11, 
12, 13, 14, 15. 

3. Do Exercise | using the insert key sequence: 20, 10, 5, 30, 40, 57, 3, 2,4, 
35, 25, 18, 22, 21. 

4. Do Exercise | using the insert key sequence: 40, 50, 70, 30, 42, 15, 20, 25, 
27, 26, 60, 55. 

5. Draw the RRr and RLr color changes that correspond to the LLr and LRr 
changes of Figure 10.16. 

6. Draw the RRb and RLb rotations that correspond to the LLb and LRb 
changes of Figure 10.17. 


11. 


33: 


14. 


Red-Black Trees 591 


Let T be a red-black tree with rank r. Write a function to compute the rank 
of each node in the tree. The time complexity of your function should be 
linear in the number of nodes in the tree. Show that this is the case. 
Compare the worst-case height of a red-black tree with n nodes and that of 
an AVL tree with the same number of nodes. 

Develop the deletion transformations for a red-black tree. Show that a 
deletion from a red-black tree requires at most one rotation. 

(a) Use the strategy described in this section to obtain a C++ function to 
compute a three-way join. Assume the existence of a function 
rebalance (X) that performs the necessary transformations if the tree 
pointer to node X is the second of two consecutive red pointers. The 
complexity of this function may be assumed to be Oflevel (X)). 

(b) Prove the correctness of your function. 

(c) What is the time complexity of your function? 

Obtain a function to perform a two-way join on red-black trees. You may 
assume the existence of functions to search, insert, delete, and perform a 
three-way join. What is the time complexity of your two-way join func- 
tion? 

Use the strategy suggested in Program 10.5 to obtain a C++ function to per- 
form the split operation in a red-black tree T. The complexity of your algo- 
rithm must be O(height(7)). Your function must work when the splitting 
key iis present in 7 and when it is not present in 7. 

Complete the complexity proof for the split operation by showing that 
whenever @ has a black child, the total work done in Step 2 of the splitting 
algorithm from initiation to the time that Q reaches the current node is 
O(r(Q)). 

Program the search, insert, and delete operations for AVL trees and red- 
black trees. 

(a) — Test the correctness of your functions. 

(b) Generate a random sequence of inserts of distinct values. Use this 
sequence to initialize each of the data structures. Next, generate a 
random sequence of searches, inserts, and deletes. In this sequence, 
the probability of a search should be 0.5, that of an insert 0.25, and 
that of a delete 0.25. The sequence length is m. Measure the time 

needed to perform the m operations in the sequence using each of the 
above data structures. 

(c) Do part (b) for n = 100, 1000, 10,000, and 100,000 and m =n, 21, and 
An, 

(d) What can you say about the relative performance of these data 


592 Efficient Binary Search Trees 


structures? 


10.4 SPLAY TREES 


We have studied balanced search trees that allow one to perform operations such 
as search, insert, delete, join, and split in O(log n) worst-case time per operation. 
In the case of priority queues, we saw that if we are interested in amortized com- 
plexity rather than worst-case complexity, simpler structures can be used. This 
is also true for search trees. Using a splay tree, we can perform the operations in 
O(log n) amortized time per operation. In this section, we develop two varieties 
of splay trees—bottom up and top down. Although the amortized complexity of 
each operation is O(log #) for both varieties, experiments indicate that top-down 
splay trees are faster than bottom-up splay trees by a constant factor. 


10.4.1 Bottom-Up Splay Trees 


A bottom-up splay tree is a binary search tree in which each search, insert. 
delete, and join operation is performed in the same way as in an ordinary binary 
search tree (see Chapter 5) except that each of these operations is followed by a 
splay. In a split, however, we first perform a splay. This makes the split very 
easy to perform. A splay consists of a sequence of rotations. For simplicity. we 
assume that each of the operations is always successful. A failure can be 
modeled as a different successful operation. For example, an unsuccesful search 
may be modeled as a search for the element in the last node encountered in the 
unsuccessful search, and an unsuccessful insert may be modeled as a successful 
search. With this assumption, the start node for a splay is obtained as follows: 


(1) Search: The splay starts at the node containing the element being sought. 

(2) Insert: The start node for the splay is the newly inserted node. 

(3) Delete: The parent of the physically deleted node is used as the start node 
for the splay. If this node is the root, then no splay is done. 

(4) ThreeWayJoin: No splay is done. 

(5) Split: Suppose that we are splitting with respect to the key i and that key i 
is actually present in the tree. We first perform a splay at the node that con- 


tains i and then split the tree. As we shall see, splitting following a splay is 
very simple. 


__ Splay rotations are performed along the path from the start node to the root 
of the binary search tree. These rotations are similar to those performed for AVL 
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trees and red-black trees. Let gq be the node at which the splay is being per- 
formed. Initially, g is the node at which the splay starts. The following steps 
define a splay: 


(1) Ifq is either 0 or the root, then the splay terminates. 


(2) If q has a parent, p, but no grandparent, then the rotation of Figure 10.19 is 
performed, and the splay terminates. 


Rp 
a —- c 
b c a b 


a, b, and c are subtrees 


Figure 10.19: Rotation when q is a right child and has no grandparent 


(3) Ifq has a parent, p, and a grandparent, gp, then the rotation is classified as 
LL (p is the left child of gp, and q is the left child of p), LR (p is the left 
child of gp, and q is the right child of p), RR, or RL. The RR and RL rota- 
tions are shown in Figure 10.20. LL and LR rotations are symmetric to 
these. The splay is repeated at the new location of g. 


Notice that all rotations move q up the tree and that following a splay, q 
becomes the new root of the search tree. As a result, splitting the tree with 
respect to a key, i, is done simply by performing a splay at / and then splitting at 
the root. Figure 10.2] shows a binary search tree before, during, and after a 
splay at the shaded node. 

In the case of Fibonacci heaps, we obtained the amortized complexity of an 
operation by using an explicit cross-charging scheme. The analysis for splay 
trees will use a potential technique. Let Pg be the initial potential of the search 
tree, and let P; be its potential following the ith operation in a sequence of m 
operations. The amortized time for the ith operation is defined to be 


(actual time for the ith operation) + P; ~ P,_1 
That is, the amortized time is the actual time plus the change in the potential. 
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Figure 10.20: RR and RL rotations 


Rearranging terms, we see that the actual time for the ith operation is 
(amortized time for the ith operation) + P;._ - Pi 
Hence, the actual time needed to perform the nm operations in the sequence is 
> (amortized time for the ith operation) + Po — Pi» 


1 
Since each operation other than a join involves a splay whose actual com- 
plexity is of the same order as that of the whole operation, and since each join 
takes O(1) time, itis sufficient to consider only the time spent performing spl2ys- 
Each splay consists of several rotations. We shall assign to each bamagtes 
fixed cost of one unit. The choice of a potential function is rather arbitrary. e 
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(d) After LR rotation 


(c) After LL rotation 


Figure 10.21: Rotations in a splay beginning at shaded node (continued on next 
page) 
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(e) After RL rotation 


Figure 10.21: Rotations in a splay beginning at the shaded node 


objective is to use one that results in as small a bound on the time complexity as 

is possible. We now define the potential function we shall use. Let the size, s (i), 

of the subtree with root i be the total number of nodes in it. The rank, r(i), of 

node i is equal to | logs (i)]. The potential of the tree is }\r(i). The potential 
é 


of an empty tree is defined to be zero. 

Suppose that in the tree of Figure 10.21(a), the subtrees a,b, «+-,j 
empty. Then (s(1), +++, 5(9)) =(9, 6, 3, 2, 1, 4, 5, 7, 8); r(3) =7 (4) = 1; 
0; and r(9) = 3. In Lemma 10.3 we use r and r’, respectively, to denote the rank 
of a node before and after a rotation. 


Lemma 10.3: Consider a binary search tree that has n elements/nodes. The 
amortized cost of a splay operation that begins at node q is at most 
3( [logan J = r(q)) + 1. 


Proof: Consider the three steps in the definition of a splay: 


(1) In this case, g either is 0 or the root. This step does not alter the potential 
of the tree, so its amortized and actual costs are the same. This cost is 1. 

(2) In this step, the rotation of Figure 10.19 (or the symmetric rotation for the 
case when q is the left child of p) is performed. Since only the ranks of P 
and q are affected, the potential change, AP, is r’(p) + r'(q)-" (p)-1Q)- 
Further, since r’(p) $ r(p), AP < r'(q)—r(q). The amortized cost of this 
step (actual cost plus potential change) is, therefore, no more than 
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rigy-r(g+l. 

(3) _ In this step only the ranks of q, p, and gp change. So, AP = r'(g) + r'(p) + 
(gp) ~ r(q) ~r(p) - r(gp). Since, r(gp) = r'(q), 
AP =r(p) + r'(gp)—r(4)- r(p) --(1) 

Consider an RR rotation. From Figure 10.20(a), we see that r‘(p) < 
°(@), rep) < rq), and r(q) S rp). So, AP < 2(r'(q)—r(g)). If r'(q) > 
1(q). AP $ 3(r'(q) — r(q))— 1. If r'(q) = r(q), then r(g) = r(q) =r) = 
r(gp). Also, sq) > s(q) + s(gp). Consequently, r'(gp) < r‘(q). To see 
this, note that if r(gp) = r‘(q), then sg) > 2° + 2° @) = 2°G! which 
violates the definition of rank. Hence, from (1), AP $ 2(r'(g)-r(q))- 1 = 
3(r°(q)— r(q))- 1. So, the amortized cost of an RR rotation is at most 
14+3(r°(q) - r(q)) - 1 = 3(r'(q) - r(q)). 

This bound may be obtained for LL, LR, and RL rotations in a similar 
way. 


The lemma now follows by observing that Steps I and 2 are mutually 
exclusive and can occur at most once. Step 3 occurs zero or more times. Sum- 
ming up over the amortized cost of a single occurrence of Steps | or 2 and all 
occurrences of Step 3, we obtain the bound of the lemma. 0 


Theorem 10.1: The total time needed for a sequence of m search, insert, delete, 
join, and split operations performed on a collection of initially empty splay trees 
is O(m log n), where n, n > 0, is the number of inserts in the sequence. 


Proof: Since none of the splay trees has more than n nodes, no node has rank 
more than | logyn J. A search (excluding the splay) does not change the rank of 
any node and hence does not affect the potential of the splay tree involved. An 
insert (excluding the splay) increases, by one, the size of every node on the path 
from the root to the newly inserted node. This causes the ranks of the nodes with 
size 2 — 1 to change. There are at most |logsn | + 1 such nodes on any insert 
path. So, each insert (excluding the splay) increases the potential by at most this 
much. Each join increases the total potential of all the splay trees by at most 
Llogon |. Deletions do not increase the potential of the involved splay tree 
except for any increase that results from the splay step. The split operation 
(excluding the splay step) reduces the overall potential by an amount equal to the 
rank of the tree just before the split (but after the splay that precedes it). So, the 
potential increase, P/, attributable to the m operations (exclusive of that attribut- 
able to the splay steps of the operations) is O(mm log n). ae 

From our definition of the amortized cost of a splay operation, it follows 
that the time for the sequence of operations is the sum of the amortized costs of 
the splays, the potential change Py - P,, and P/. From Lemma 10.3, it follows 
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that the sum of the amortized costs is O(m logn). The initial potential, Po, is 0, 
and the final potential, P,,,, is >0. So, the total time is O(n logn). O 


10.4.2 Top-Down Splay Trees 


As in the case of a bottom-up splay tree, a ThreeWayJoin is implemented the 
same in a top-down splay tree as in Section 5.7.5. For the remaining operations, 
let the splay node be as defined for a bottom-up splay tree. For each operation, 
we follow a path from the root to the splay node as in Section 5.7.5. However, as 
we follow this path, we partition the binary search tree into three components—a 
binary search tree small of elements whose key is less than that of the element in 
the splay node, a binary search tree big of elements whose key is greater than 
that of the element in the splay node, and the splay node. Notice that in this 
downward traversal of the path from the root to the splay node, we do not actu- 
ally know which node is the splay node until we get to it. So, the downward 
traversal is done using the key & associated with the operation that is being per- 
formed. 

For the partitioning, we begin with two empty binary search trees small and 
big. It is convenient to give these trees a header node that is deleted at the end. 
Let s and b, respectively, be initialized to the header nodes of small and big. The 
downward traversal to the splay node begins at the root. Let x denote the node 
we currently are at. We begin with x being the wee root. There are 7 cases to 


consider: 


Case 0: x is the splay node. 
Terminate the partitioning process. 

Case L: The splay node is the left child of x. 
In this case, x and its right subtree contain keys that are greater than 
that in the splay node. So, we make x the left child of b 
(b—>leftChild = x) and set b =x and x = x-/leftChild. Notice that this 
automatically places the right subtree of x into big. Figure 10.22 
shows a schematic for this case. 

Case R: The splay node is the right child of x. 
This case is symmetric to Case L. Now, x and its left subtree contain 
keys that are less than that in the splay node. So, we make x the right 
child of s (s—rightChild = x) and set s =x and x = x-srightChild. 
Notice that this automatically places the left subtree of x into small. 


Case LR:The splay node is in the right subtree of the left child of x. 
This case is handled as Case L followed by Case R. 


Case RL:The splay node is in the left subtree of the right child of x. 
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(b) After L transformation 


Figure 10.22: Case L fora top-down splay tree 


This case is handled as Case R followed by Case L. 

Case LL: The splay node is in the left subtree of the left child of x. 
This case is not handled as two applications of Case L. Instead we 
perform an LL rotation around x. Figure 10.23 shows a schematic for 
this case. The shown transformation is accomplished by the following 
code fragment: 


bleftChild = x leftChild; 
b = bleftChild; 
x-leftChild = b—vrightChild; 
b—rightChild =x; 
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(a) Before LL transformation 


CL Cr 


(b) After LL transformation 


Figure 10.23: Case LL for a top-down splay tree 


x= bleftChild ; 


Case RR:The splay node is in the right subtree of the right child of x. 
This case is symmetric to Case LL. 


The above transformations are applied repeatedly until terminated ae 
application of Case 0. Upon termination, x is the splay node. Now, the 
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subtree of x is made the right subtree of s and the right subtree of x is made the 
left subtree of b. Finally, the header nodes of the small and big trees are defeted. 

In case we were performing a split operation and x contains the split key, 
we retum small, x—>data, and big as the result of the split. For a search, insert 
and delete, we make small and big, respectively, the left and right subtrees of x 
and the tree rooted at x is the new binary search uee (we assume that in the 
downward quest for the splay node, the remaining tasks associated with the 
search, insert and delete operations have been done). 


Example 10.5: Suppose we are searching for the key 5 in the top-down splay 
tree of Figure 10.21(a). Although we don’t know this at this time, the splay node 
is the shaded node. The path from the root to the splay node is determined by 
comparing the search key 5 with the key in the current node. We start with the 
current node pointer x at the root and two empty splay trees—smalf and big. 
These empty splay trees have a header node. The variables s and 6, respectively, 
point to these header nodes. Since the splay node is in the left subtree of the 
right child of x, an RL transformation is called for. The search tree as well as the 
cee small and big following the RL tansformation are shown in Figure 
10.24(a). 

Now, since the splay node is in the right subtree of the left child of the new 
x, an LR transformation is made and we obtain the configuration of Figure 
10.24(b). Next, we make an LL wansformation (Figure 10.24(c)) and an RR 
transformation (Figure 10.24(d)). Now, x is at the splay node. The left subtree of 
x is made the right subtree of s and the right subtree of x is made the left subtree 
of b (Figure 10.24(e)). Finally, we delete the header nodes and make the small 
and big trees subtrees of x as shown in Figure 10.24(f).0 


EXERCISES 


1. Obtain figures corresponding to Figures 10.19 and 10.20 for the symmetric 
bottom-up splay tree rotations. 

2. What is the maximum height of a bottom-up splay tree that is created as the 
result of n insertions made into an initially empty splay tree? Give an 
example of a sequence of inserts that results in a splay tree of this height. 

3. Complete the proof of Lemma 10.3 by providing the proof for the case of 
an RL rotation. Note that the proofs for LL and LR rotations are similar to 
those for RR and RL rotations, respectively, as the rotations are symmetric. 

4. Explain how a two-way join should be performed in a bottom-up splay tree 
so that the amortized cost of each splay tree operation remains O(log 7). 

5. Explain how a split with respect to key / is to be performed when key / is 
not present in the bottom-up splay tee. The amortized cost of each 
bottom-up splay tree operation should be Oflog ). 
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small 


a > 


(b) After LR transformation 


Figure 10.24: Example for top-down splay tree (continued on next page) 


6. Implement the bottom-up splay tree data structure as a C++ class. Test all 
functions using your own test data. 

7. [Sleator and Tarjan) Suppose we modify the definition of s(/) used in cor 
nection with the complexity analysis of bottom-up splay trees. et all 
node i have a positive weight p(i). Let s (i) be the sum of the weights 
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(d) After RR transformation 
*—@ small 
a 
s 
-_ 
b 
3 e 
c d 
(e) After moving subtrees of splay node 


Figure 10.24: Example for top-down splay tee (continued on next page) 
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(f) Final search tree 


Figure 10.24: Example for top-down splay tree 


nodes in the subtree with root i. The rank of this subtree is log>s (i). 


(a) Let f be the root of a splay tree. Show that the amortized cost of a 
splay that begins at node q is at most 3(r(t) — r(q)) + 1, where r is 
the rank just before the splay. 


(b) Let S be a sequence of n inserts and m searches. Assume that each of 
the n inserts adds a new element to the splay tree and that all 
searches are successful. Let p(i), p{i) > 0, be the number of times 
element i is sought. The p(é)’s satisfy the following equality: 


Letij=m 


i=l 


Show that the total time spent on the mm searches is 


O(m + Sp (i)log (n/p (i))) 


n 
Note that since Qn + Yp(i)log(n/p@)) is an information 
i=l 
theoretic bound on the search time in a static search tree (the optimal 
binary search tree of Section 10.1 is an example of such a tree), 
bottom-up splay trees are optimal to within a constant factor for the 
representation of a static set of elements. 
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8. Obtain figures corresponding to Figures 10.22 and 10.23 for the top-down 
splay tree transformations R, RR, RL, and LR. 

9. What is the maximum height of a top-down splay tree that is created as the 
result of n insertions made into an initially empty splay tree? Give an 
example of a sequence of inserts that results in a splay tree of this height. 


10. Implement the top-down splay tree data structure as a C++ class. Test all 
functions using your own test data. 


10.5 REFERENCES AND SELECTED READINGS 
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of data structures and applications,” edited by D. Mehta and S. Sahni, Chapman 
& Hall/CRC, Boca Raton, 2005. 
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Multiway Search Trees 


11.1 m-WAY SEARCH TREES 


111.1 Definition and Properties 


Balanced binary search trees such as AVL and red-black trees allow us to search. 
insert, and delete in O(log 7) time, where n is the number of elements. While 
this may seem to be a remarkable accomplishment, we can improve the perfor- 
mance of search structures by capitalizing on the exhorbitant ume it takes to 
make a memory access (whether to main memory or to disk) relauve to the ume 
it takes to perform an arithmetic or logic operation in a modem computer. An 
access to main memory typically takes approximately 100 tmes the me to do 
an arithmetic operation while an access to disk takes about 10.000 umes the ume 
for an arithmetic operation. Because of this significant mismatch between pro- 
cessor speed and memory access time. data is typically moved from main 
memory to cache (fast memory) in units of a cache-line size (of the order of 100 
bytes) and from disk to main memory in units of a block (several xi!o bytes). 
For uniformity with disks, we say that main memon is organized into blocks: the 
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size of each block being equal to that of a cache line. AVL and red-black trees 
are unable to take advantage of this large unit (i.e., block) in which data is 
moved from slow memory (main or disk) to faster memory (cache or main) since 
the node size is typically only a few bytes. Consider an AVL tree with 1,000,000 
elements. It’s height may be as much as [1.44log>(n+2)), which is 28. To search 
this tree for an element with a specified key, we must access those nodes that are 
on the search path from the root to the node that contains the desired element. 
This path may contain 28 nodes and if each of these 28 nodes lies in a different 
memory block, a totat of 28 memory accesses and 28 compares are made in the 
worst case. Most of the search time is spent on memory access! To improve per- 
formance, we must reduce the number of memory accesses. Notice that if haly- 
ing the number of memory accesses resulted in a doubling of the number of com- 
parisons, we would still achieve a reduction in total search time, Since the 
number of memory accesses is closely tied to the height of the search uee, we 
must reduce tree height. To break the log>(n +1) barrier on tree height resulting 
from the use of binary search trees, we must use search trees whose degree is 
more than 2. In practice, we use the largest degree for which the tree node fits 
into a block (whether cache line or disk block). 


Definition: An m-way search tree is either empty or satisfies the following pro- 
petties: 


(1) The root has at most m subtrees and has the following structure: 
n, Ag (E, Aq), (En A2). °° (Ean) 


where the A;,OSiSn<m, are pointers to subtrees, and the 
E;, 1 <i Sn <m, are elements. Each element £; has a key E,.K. 

(2) EK <Ejy).K.Usi<cn 

(3) Let E9.K = ~0e and E,4;.K =e. All keys in the subtree A, are less than 
E; 44.K and greater than E,.K,0<Si Sa. 

(4) The subtrees A;, 0 <i <x, are also m-way search trees. O 


We may verify that binary search trees are two-way search trees, A three- 
way search tree is shown in Figure 11.1. For convenience, only keys are shown 
in this figure as well as in all remaining figures in this chapter. 

In a tree of degree m and height h, the maximum number of nodes is 

XL om! =(n"-1tm-1) 
Osigh-1 
Since each node has at most m — | elements, the maximum number of elements 
in an m-way tree of height A is m" — 1, Fora binary tree with h = 3 this quantity 
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seeee 
es 40) node | schematic format | 


iu a | 2,b, (20,), (40, d) 

eee b 2, 0, (10, 0), (15, 0) 

(10, 15) S » 50) | c¢ | 2,0,(25,e), (30,0) 

Se d 2, 0, (45, 0), (SO, 0) 
e 1, 0, (28, 0) 


Figure 11.1: Example of a three-way search tree 


is 7. For a 200-way tree with A = 3 we have m’ — 1 = 8 * 106-1. 

To achieve a performance close to that of the best m-way search trees for a 
given number of elements n, the search tree must be balanced. The particular 
varieties of balanced m-way search trees we shall consider here are known as B- 
trees and B “-trees. 


11.1.2 Searching an m-Way Search Tree 


Suppose we wish to search an m-way search tree for an element whose key is x. 
We begin at the root of the tree. Assume that this node has the structure given in 
the definition of an m-way search tree. For convenience, assume that E.K = —0o 
and E,,,,.K = +0. By searching the keys of the root, we determine i such that 
E,.K <x < E;,,.K. If x = £;.K, then the search is complete. If x 4 £;.K, then 
from the definition of an m-way search tree, it follows that if x is in the tree, it 
must be in subtree A;. So, we move to the root of this subtree and proceed to 
search it. This process continues until either we find x or we have determined 
that x is not in the tree (the search leads us to an-empty subtree), When the 
number of elements in the node being searched is small, a sequential search is 
used. When this number is large, a binary search may be used. A high-level 
description of the algorithm to search an m-way search tree is given in Program 
i. 
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# Search an m-way search tee for an element with key x. 
# Return the element if found. Rewm NULL otherwise. 
Eo.K = - MAXKEY; 

for (*p = root; p; p=A,) 


Let p have the format n, Ao, (E}, 41), +++. (Ey. Ag) 
En 41-K = MAXKEY; 
Determine i such that E;.K $x < E;,).K; 
if (x == E;.K) return E;; 
} 
“4 x is not in the tree 
return NULL; 


Program 11.1: Searching an m-way search tree 


EXERCISES 


1. Draw a sample 5-way search tree. 

2. What is the minimum number of elements in an m-way search tree whose 
height is A? 

3. Write an algorithm to insert an element whose key is x into an m-way 
search tree. What is the complexity of your algorithm? 

4. Write an algorithm to delete the element whose key is x from an m-way 
search tree. What is the complexity of your algorithm? 


18.2  B-TREES 
11.2.1 Definition and Properties 


In defining a B-tree, it is convenient to extend m-way search trees by the addition 
of external nodes. An external (or failure) node is added wherever we otherwise 
have a NULL pointer. An external node represents a node that can be reached 
during a search only if the element being sought is not in the tree. Nodes that are 
not external nodes are called internal nodes. 


Definition: A B-tree of order m is an m-way search tree that either is empty or 
satisfies the following properties: 
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(1). The root node has at least two children. 


(2) All nodes other than the root node and external nodes have at least fm /2| 
children. 
(3) All external nodes are at the same level. O 


Observe that when i = 3, all internal nodes of a B-tree have a degree that 
is either 2 or 3 and when m = 4, the permissible degrees for these nodes are 2, 3 
and 4, For this reason, a B-tree of order 3 is known as a 2-3 tree and a B-tree of 
order 4 is known as a 2-3-4 tree. A B-tree of order 5 is not a 2-3-4-5 tree as a B- 
tree of order 5 cannot have nodes whose degree is 2 (except for the root). Also, 
notice that all B-trees of order 2 are full binary trees. Hence, B-trees of order 2 
exist only when the number of key values is 2*— 1 for some k. However, for any 
n 2Oand any m > 2, there is always a B-tree of order m that contains 1 keys. 

Figure 11.2 shows a 2-3 tree (ie., a B-tree of order 3) and Figure 11.3 
shows a 2-3-4 tree (i.e., a B-tree of order 4), Notice that each (internal) node of a 
2-3 tree can hold 2 elements while each such node of a 2-3-4 tree can hold 3 ele- 
ments. In the figures, only the keys are shown. Note also that although Figures 
11.2 and 11.3 show external nodes, external nodes are introduced only to make it 
easier to define and talk about B-trees. External nodes are not physically 
represented inside a computer. Rather, the corresponding child pointer of the 
parent of each external node is set to NULL. 


wo 
io) 


10|20 80] 


a 


Figure 11.2: Example of a 2-3 tree 
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Figure 11.3: Example of a 2-3-4 tree 


11.2.2. Number of Elements in a B-Tree 


A B-tree of order m in which all extemal nodes are at Jevel /+1 has at most 
m'—1 keys. What is the minumum number, N, of elements in such a B-tree? 
From the definition of a B-tree we know that if / > 1, the root node has at least 
two children. Hence, there are at least two nodes at level 2. Each of these nodes 
must have at least [m/2] children. Thus, there are at least 2{m/2] nodes at 
level 3. At level 4 there must be at least 2[m/2}? nodes, and continuing this 
argument, we see that there are at least 2[m/2]'~* nodes at level ! when / > 1. 
All of these nodes are internal nodes. If the keys in the tree are K,,K2, °°+, Ky 
and Kj < K;4), 1 $i</N, then the number of external nodes is V + 1. This is so 
because failures occur for K; <x < Kj), 0Si $M, where Ko =—<» and Ky,1 = 
too, This results in N + I different nodes that one could reach while searching 
for a key x that is not in the B-tree. Therefore, we have 


N +1 = numberof external nodes 
= number of nodes at level {/ + 1) 
> 2Jm/2}"" 


soN 2 2fm/2}! -1, 121 ; . 
This in turn implies that if there are N elements (equivalently, keys) in a 


B-tree of order m, then all intemal nodes are at levels less than or equal to /, 
1S logtys21 {(N + 1)/2] + L. Ifa B-tree node can be examined with a single 
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memory access, the maximum number of accesses that have to be made for a 
search is /. Using a B-tree of order m = 200, which is quite practical for a disk 
resident B-tree, a tree with N < 2x 10° —2 will have / < logyo9 ((N + 1)/2) +1. 
Since / is an integer, we obtain / <3. ForN <2x 108 — 2 we get <4, 

To search a B-tree with a number of memory accesses equal to the B-tree 
height we must be able to examine a B-tree node with a single memory access, 
This means that the size of a node should not exceed the size of a memory block 
(i.e., size of a cache line or disk block). For main-memory resident B-trees an m 
in the tens is practical and for disk resident B-trees an sn in the hundreds is prac- 
tical. 


11.2.3 Insertion into a B-Tree 


The insertion algorithm for B-trees first performs a search to determine the leaf 
node, p, into which the new key is to be inserted. If the insertion of the new key 
into p results in p having m keys, the node p is split. Otherwise, the new p is 
written to the disk, and the insertion is complete. To split the node, assume that 
following the insertion of the new element, p has the format 


m, Ag, (E,A1), °**, (EmAm), and E; < Ej, 1Si<m 


The node is split into two nodes, p and q, with the following formats: 
node p: [m/2] -1, Ao, (E\Ay), «+. (Efins2y-1-A fmsa}-1) (LS) 
node g: m — [m/2], A fns2]> E fn2}41A (m2}eis °° *y EmvAm) 


The remaining element, E ,,/2), and a pointer to the new node, g, form a tuple 
(E n/2] 4). This is to be inserted into the parent of p. 

Inserting into the parent may require us to split the parent, and this splitting 
process can propagate all the way up to the root. When the root splits, a new 
root with a single element is created, and the height of the B-tree increases by 
one. A high-level description of the insertion algorithm for a disk resident B-tree 
is given in Program 11.2. 


Example 11.1: Consider inserting an element with key 70 into the 2-3 tree of 
Figure 11.2. First we search for this key. If the key is already in the tree, then 
the existing element with this key is replaced by the new element. Since 70 is 
not in our example 2-3 tree, the new element is inserted and the total number of 
elements in the tree increases by 1. For the insertion, we need to know the leaf 
node encountered during the search for 70. Note that whenever we search for a 
key that is not in the 2-3 tree, the search encounters a unique leaf node. The leaf 
node encountered during the search for 70 is the node C, with key 80. Since this 
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# Insert element x into a disk resident B-tree. 
Search the B-tree for an element E with key 1X, 
if such an £ is found, replace E with x and return; 
Otherwise, let p be the leaf into which «x is to be inserted; 
q=NULL; 
for (e = x; p; p = p—>parent()) 
{4 Ce, g) is to be inserted into p 
Insert (e, 7) into appropriate position in node p; 
Let the resulting node have the form: 1, Ag. (E;, At) °**« (En Audi 
if (n <= m — 1) { / resulting node is not too big 
write node p to disk; return; 


} 
H node p has to be split 
Let p and q be defined as in Eq. (11.5); 
€=Eims2)3 
: write nodes p and g to the disk; 
// a new root is to be created 
Create a new node r with format 1, root, (¢, 9); 
root=r, 
write root to disk; 


Program 11.2: Insertion into a B-tree 


node has only one element, the new element may be inserted here. The resulting 
2-3 tree is shown in Figure 11.4(a). : 

Next, consider insesting an element with key 30. This time the search 
encounters the leaf node B. Since B is full, it is necessary to split B. For this, we 
first symbolically insert the new element into B to get the key sequence 10, 20, 
30. Then the overfull node is split using Eq. 11.5. Following the split, B has the 
key sequence 10 and the new node, D, has 30. The middle element, whose key is 
20, together with a pointer to the new node D is inserted into the parent A of B. 


The resulting 2-3 tree is shown in Figure 11.4(b). Ff ‘ 

Finally, consider the insertion of an element with key 60 into the 23 tree of 
Figure 11.4(b). The leaf node encountered during the search for 60 is node c 
Since C is full, a new node, E, is created. Node E contarns the element with the 
largest key (80). Node C contains the element with the parte key ss site 
element with the median key (70), together Label mae e pe e , is 
to be inserted into the parent A of C. Again, se re ; c Econ 
taining the element with the largest key among (20, 40, 70) is created. Ay 
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40] | 20} 40 


bt at ] Cj iil 


(a) 70 inserted (b) 30 inserted 


Figure 11.4: Insertion into the 2-3 tree of Figure 11.2 


before, A contains the element with the smallest key. B and D remain the left 
and middle children of A, respectively, and C and E become these children of F. 
if A had a parent, then the element with the median key 40 and a pointer to the 
new node, F, would be inserted into this parent node. Since A does not have a 
parent, we create a new root, G, for the 2-3 tree. Node G contains the element 
with key 40, together with child pointers to A and F. The new 2-3 tree is as 
shown in Figure 11.5. 0 


Analysis of B-tree Insertion: For convenience, assume the B-tree is disk 
resident. If h is the height of the B-tree, then A disk accesses are made during the 
top-down search. In the worst case, all # of the accessed nodes may split during 
the bottom-up splitting pass. When a node other than the root splits, we need to 
write out two nodes. When the root splits, three nodes are written out. If we 
assume that the A nodes read in during the top-down pass can be saved in 
memory so that they are not to be retrieved from disk during the bottom-up pass, 
then the number of disk accesses for an insertion is at most # (downward pass} + 
2(h — 1) (nonroot splits) + 3 (root split) = 3h + 1. 

The average number of disk accesses is, however, approximately A + 1 for 
large m. To see this, suppose we start with an empty B-tree and insert N values 
into it. The total number of nodes split is at most p — 2, where p is the number of 
internal nodes in the final B-tree with N entries. This upper bound of p — 2 fol- 
lows from the observation that each time a node splits, at least one additional 
node is created. When the root splits, two additional nodes are created. The first 
node created results from no splitting, and if a B-tree has more than one node, 
then the root must have split at least once. Figure 11.6 shows that p — 2 is the 
tightest upper bound on the number of nodes split in the creation of a p-node B- 
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Figure 11.5; Insertion of 60 into the 2-3 wee of Figure 11.4(b) 


tree when p > 2 (note that there is no B-tree with p = 2). A B-tree of order m 
with p nodes has at least 1 + ([m/2] - 1)(p ~ 1) keys, as the root has at least one 
key and remaining nodes have at least [m/2]-1 keys each. The average 


number of splits, s4,,, may now be determined as follows: 


Sarg = (total number of splits 
<(p -2V{1 + ({m2]-Dep - D} 
<lA{m2] - 1) 


For m = 200 this means that the average number of node splits is less than 1/99 
per key inserted. The number of disk accesses in an insertion is h + 25-1, 
where 5 is the number of nodes that are split during the insertion. So, the average 
number of disk accesses ish + 2sayg + 1<h + 101A9=h+1.0 


11.2.4 Deletion from a B-Tree 


For convenience, assume we are deleting from a B-tree dhat resides‘on disk: 
Suppose we are to delete the element whose key is x. First, we search for this 
key. If x is not found, no element is to be deleted. If. is found ms node, z, that 
is not a leaf, then the position occupied by the cores ane wg is 
filled by an element from a leaf node of the B-tree. Suppose that x is the ith key 
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(a)p=l,s=0 


(c)p=4,s=2 


Figure 11.6: B-trees of order 3 


in z (i.e., x = E;.K). Then E; may be replaced by either the element with smallest 
key in the suburee A; or the element with largest key in the subtree A;_. Both of 
these elements are in leaf nodes. In this way the deletion from a nonleaf node is 
transformed into a deletion from a leaf. For example, if we are to delete the ele- 
ment with key 20 that is in the root of Figure 11.6 (c), then this element may be 
feplaced by either the element with key 10 or the element with key 25. Both are 
in leaf nodes. Once the replacement is done, we are faced with the problem of 
deleting either the 10 or the 25 from a leaf. 

There are four possible cases when deleting from a leaf node p. In the first, 
p is also the root. If the root is left with at least one element, the changed root is 
written to disk and we are done. Otherwise, the B-tree is empty following the 
deletion. In the remaining cases, P is not the root. In the second case, following 
the deletion, p has at least [2/2] — 1 elements. The modified leaf is written to 
disk, and we are done. 

In the third case (rotation), p has [m/2]-2 elements, and its nearest 


B-Trees 617 


sibling, g, has at teast [m/2] elements. To determine this, we examine only one 
of the two (at most) nearest siblings that p may have. p is deficient, as it has one 
less than the minimum number of elements required. ¢ has more elements than 
the minimum required. A rotation is performed. In this rotation, the number of 
elements in g decreases by one, and the number in p increases by one. As a 
result, neither p nor q is deficient following the rotation. The rotation leaves 
behind a valid B-tree. Let r be the parent of p and g. If g is the nearest right 
sibling of p, then let i be such that £; is the ith element in r, all elements in p 
have a key that is less than E;.X, and all those in q have a key that is greater than 
E,.K. For the rotation, E; becomes the rightmost element in p, E, is replaced, in 
r, by the first (i.e., smallest) element in g, and the leftmost subtree of g becomes 
the rightmost subtree of p. The changed nodes p,q, and r are written to disk, and 
the deletion is complete. The case when g is the nearest left sibling of p is simi- 


lar. 
Figure 11.7 shows the rotation cases for a 2-3 wee. A “‘?"" denotes 2 situa- 
tion in which the presence or absence of an clement is ielevant. a, b,c, and d 
denote the children (i.e., roots of subtrees) of nodes. 
In the fourth case (combine) for deletion, p has [m/2] ~ 2 elements, and 
its nearest sibling g has {m/2]-1 elements. So, p is deficient, and g has the 
minimum number of elements required by a nonroot node. Now, nodes p and ¢ 
and the in-between element E; in the parent r are combined to form a single 
node. The combined node has ({m/2] - 2) + ({m/2] - 1) + 1 =2[m/2] -2 
m — | elements, which will, at most, fill the node. The combined node is written 
to disk. The combining operation reduces the number of elements in the parent 
node, r, by one. If the parent does not become deficient (ie., it has at least one 
element if it is the root and at least [m/2] — 1 elements if it is not the root), the 
changed parent is written to disk, and we are done. Otherwise, if the deficient 
Parent is the root, it is discarded, as it has no elements. If the deficient parent is 
not the root, it has exactly [m/2] -2 elements. To remove this deficiency, we 
first attempt a rotation with one of r’s nearest siblings. If this is not possible, a 
combine is done. This process of combining can continue up the B-tree only 
until the children of the root are combined. ease i 
Figure 11.8 shows the two cases for a combine in a 2-3 mee when p is the 
left child of r. We leave it as an exercise ee obtain the figures for the cases when 
Pisa middle child and whenpisatightchild. = 
A high-level description of the deletion algorithm is provided in Program 
11.3, 
ean i 1.{a). Suppose that 
Example 11.2: Let us begin with the a toes of nae Ve oe Spiers za 
the two element fields in a node of a 2 
. ly delete this element from node C. 
delete the element with key 70, we must merely the element with key 10 from 
The result is shown in Figure 11-9(b). To delete 
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(c) p is the right child of r 


Figure 11,7: The three cases for rotation in a 2-3 tree 


the 2-3 tree of Figure 11.9(b), we need to shift dataR to data in node B. This 
results in the 2-3 tree of Figure 11.9(c). 


Next consider the deletion of the element with key 60. This leaves node C 
deficient. Since the right sibling, D, of C has 3 elements, we are in case 3 and a 
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Figure 11.8: Combining in a 2-3 tree when pis the left child of r 


rotation is performed. In this rotation, we move the the in-between element (i.¢., 
the element whose key is 80) of the parent A of C and D to the datal. posttion of 
C and move the smallest element of D (i.e., the element whose key is 20) into the 
in-between position of the parent A of C and D (i.e. the dataR position of A). 
The resulting 2-3 tree takes the form shown in Figure 11.9(d). When the element 
with key 95 is deleted, node D becomes deficient. The rotation performed when 
the 60 was deleted is not possible now, as the left sibling, C. has the minimum 
number of elements required by a node in a B-tree of order 3. We now are in 
case 4 and must combine nodes C and D and the in-between element (90) in the 
parent A of C and D. For this, we move the 90 into the Jeft sibling, C, and delete 
node D. Notice that in a combine, one node is deleted, whereas in a rotation, no 
node is deleted. The deletion of 95 results in the 2-3 tree of Figure 11.9(e). 
Deleting the element with key 90 from this tree results in the 2-3 tree of Figure 
11.9(f). Now consider deleting the element with key 20 from this tee. Node B 
becomes deficient. At this time, we examine B's right sibling, C. If C has excess 
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# Delete element with key x. 
Search the B-tree for the node p that contains the element whose key is x; 
if there is no such p return; // no element to delete 
Let p be of the form n, Ag, (E),A,), °° +, (Ey,An) and E;.K = x3 
if p is nota leaf { 
Replace £; with the element with the smallest key in subtree A;; 
Let p be the leaf of A; from which this smallest element was taken; 
Let p be of the formn, Ag, (EyA1), °° + (EnAn)s 
i=l; 
} 
# delete E; from node p, a leaf 
Delete (E;, A;) from p; n--; 
while ((n < [m/2] - 1) && p != root) 
if p has a nearest right sibling q { 
Let q: tg, Ag, (Ef, A), «--, (ER, A4)s 
Let r:n,, Ag, (Ej, Ai), ***, (Ey, Af,) be the parent of p and g; 
Let Aj = q and Aj_; =p; 
if (n, > = [m/2]) {// rotation 
(Ens1. An et) = (Ej, 48); 2 =n + 1; update node p 
Ej = Ef; /! update node r 
(4g, AB, (Ef, AY), +=) = (gh, AD, (E48, AB), °° 5 


H update node g 
write nodes p, q and r to disk; return, 
} 4 end of rotation 
A combine p, Ej, and q 
$= 2* [m2] -2; 
writes, Ao, (Ey, Ay), “°° s (Env An)s (Ej, 48), (EF. AV). «++ (EB, AR) 
to disk as node p; 


# update for next iteration 
(n, Ag+) = (4-1, Ab + (Eft AG -1)s (Ejay Afni) 0+) 
per 
} //end of if p has a nearest right sibling 
else {// node p must have a left sibling 
Hf this is symmetric to the case where p has a right sibling, 
Hand is left as an exercise 
} 4 end of if-else and while 
if (n) write p: (n, Ag, +++. (Ey An)) 
else root = Ay; // new root 


Program 11.3: Deletion from a B-tree that resides on disk 
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(c) 10 deleted 


Figure 11.9: Deletion from a 2-3 tree (continued on next page) 
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(g) 20 deleted 


(f) 90 deleted 


Figure 11.9: Deletion from a 2-3 tree 


elements, we can perform a rotation similar to that done during the deletion of 
60. Otherwise, a combine is performed, Since C doesn’t have excess elements, 
we proceed in a manner similar to the deletion of 95 and do a combine. This 
time the elements with keys 50 and 80 are moved into B, and node C is deleted. 
This, however, causes the parent node A to become deficient. If the parent had 
not been a root, we would examine its left or right sibling, as we did when nodes 
C (deletion of 60) and D (deletion of 95) became empty. Since A is the root, it is 
simply deleted, and B becomes the new root (Figure 11.9(g)). Recall that a root 
is deficient iff it has no element. O 


Analysis of B-tree Deletion: Once again, we assume a disk-resident B-tree and 
that disk nodes accessed during the downward search pass may be saved in a 
stack in main memory, so they do not need to be reaccessed from the disk during 
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the upward restructuring pass. For a B-tree of height h, A disk accesses are made 
to find the node from which the key is to be deleted and to transform the deletion 
to a deletion from a leaf. In the worst case, a combine takes place at each of the 
last A ~ 2 nodes on the root-to-leaf path, and a rotation takes place at the second 
node on this path. The h-2 combines require this many disk accesses to 
retrieve a nearest sibling for each node and another A ~ 2 accesses to write out 
the combined nodes. The rotation requires one access to read a nearest sibling 
and three to write out the three nodes that are changed. The tota? number of disk 
accesses is 3h. 

The deletion time can be reduced at the expense of disk space and a slight 
increase in node size by including a delete bit, F;, for each element, E;, in a 
node. Then we can set F; = J if E; has not been deleted and F; = 0 if ithas. No 
physical deletion takes place. In this case a delete requires a maximum of h + 1 
accesses (h to Jocate the node containing the element to be deleted and 1 to write 
out this node with the appropriate delete bit set to 0), With this strategy, the 
number of nodes in the tree never decreases. However, the space used by 
deleted entries can be reused during further insertions (see Exercises). As a 
result, this strategy has little effect on search and insert times (the number of lev- 
els increases very slowly when mis large). The time taken to insert an item may 
even decrease slightly because of the ability to reuse deleted element space. 
Such reuses would not require us to split any nodes. O 


EXERCISES 

1. Show that all B-trees of order 2 are full binary trees. 

2. Use the insertion algorithm of Program 11.2 to insert an element with key 
40 into the 2-3 tree of Figure 11.9{a). Show the resulting 2-3 tree. 

3. Use the insertion algorithm of Program 11.2 to insert elements with keys 
45, 95, 96, and 97, in this order, into the 2-3-4 tree of Figure 11.3. Show the 
resulting 2-3-4 tree follwoing each insert. 

4. Use the deletion algorithm of Program 11.3 to delete the elements with 
keys 90, 95, 80, 70, 60 and 50, in this order, from the 2-3 tree of Figure 
11.9(a). Show the resulting 2-3 tree following each deletion. 

5. Use the deletion algorithm of Program 11.3 to delete the elements with 
keys 85, 90, 92, 75, 60, and 70 from the 2-3-4 tree of Figure | 1.3. Show the 
resulting 2-3-4 tree following each deletion. 

6. (a) Insert elements with keys 62, 5, 85, and 75 one at a time into the 
order-5 B-tree of Figure 11.10. Show the new wee after each ele- 
ment is inserted. Do the insertion using the insertion process 
described in the text. 

(b) Assuming that the tree is kept on a disk and one node may be 
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"ofis} || 


3000 


Figure 11.10: B-tree of order 5 


retrieved at a time, how many disk accesses are needed to make each 
insertion? State any assumptions you make. 

(c) Delete the elements with keys 45, 40, 10, and 25 from the order-5 B- 
tree of Figure 11.10. Show the tree following each deletion. The 
deletions are to be performed using the deletion process described in 
the text. 


(d) How many disk accesses are made for each of the deletions? 


Complete Figure 11.8 by adding figures for the cases when p is the middle 
child and right child of its parent. 


Complete the symmetric case of Program 11.3. 


Develop the C++ class TwoThreeTree, which implements a 2-3 tree. You 
must include functions to search, insert and delete. Test your class using 
your own test data. 

Develop the C++ class TwoThreeFourTree, which implements a 2-3-4 tree. 
You must include functions to search, insert and delete. Test your class 
using your own test data. 


Write insertion and deletion algorithms for B-trees assuming that with each 
element is associated an additional data member, deleted, such that 
deleted = false iff the corresponding element has not been deleted, Dele- 
tions should be accomplished by setting deleted = false, and insertions 


should make use of deleted space whenever possible without restructuring 
the tree. 


12. 


14, 


15. 


16. 
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Write algorithms to search and delete keys from a B-tree by position; that 
is, Get(k) finds the kth smailest key, and Delete (k) deletes the kth smallest 
key in the tree. (Hint: To do this efficiently, additional information must 
be kept in each node. With each pair (E;,A;) keep N, = uy {number of 
elements in the subtree A; + 1).) What are the worst-case computing times 
of your algorithms? 

The text assumed a node stnicture that was sequential. However, we need 
to perform the following functions on a B-tree node: search, insert, delete, 
join, and split. 

(a) Explain why each of these functions is important during a search, 
insert, and delete operation in the B-tree. 

(b) Explain how a red-black tree could be used to represent each node. 
You will need to use integer pointers and regard each red-black tree 
as embedded in an array. 

{c) What kind of performance gain/loss do you expect using red-black 
trees for each node instead of a sequential organization? Try to 
quantify your answer. 

Modify Program 11.2 so that when node p has m elements, we first check 10 
see if either the nearest left sibling or the nearest right sibling of p has 
fewer than m — | elements. If so, p is not split. Instead, a rotation is per- 
formed moving either the smallest or largest element in p to its parent. The 
corresponding element in the parent, together with a subtree, is moved fo 
the sibling of p that has space for another element. 

(Bayer and McCreight} Suppose that an insertion has been made into 

node p and that it has become over-full (i.¢., it now contains m elements), 

Further, suppose that ps nearest right sibling q is full (i.e., it contains m ~ 

elements). So, the elements in p and g together with the in-between ele- 

ment in the parent of p and q make 2m elements. These 2m elements may 
be partitioned into three nodes p, q and rf containing 

[(2m ~ 2)/3}, |(2m ~ 1)/3], and |2m/3] elements, respectively, plus two 

in-between elements {one for p and q and the other for ¢ and n. So, we 

may split p and q into 3 nodes p,q, and r that are almost two-thirds full, 
replace the former in-between element for p and q with the new one, and 
then insert the in-between element for g and r together with a pointer to the 
new node r into the parent of p and g. The case when q is the nearest left 


sibling of p is similar. 
Rewrite Program 11.2 so that node splittings occur only as described here. 
A B*tree of order m is a search tree that either is emply or satisfies the fol- 


lowing properties: 
(a) The root node has at least two and at most 2|(2m ~2)/3] +1 
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children. 


(b) The remaining interna! nodes have at most m and at least 
{(2m — 1)/3] children each. 


(c) All external nodes are on the same level. r 

For a B*-tree of order m that contains N elements, show that if 

x = [(2m—-1)/3}, then 

{a) _ the height, A, of the B*-tree satisfies A <1 + log, {(N + 1)/2} 

(b) the number of nodes p in the B*-tree  satisifies 
PS1+(N-1)/@- 1) 

‘What is the average number of splits per insert if a B*-tree is built up start- 

ing from an empty B*-tree? 

Using the splitting technique of Exercise 15, write an algorithm to insert a 

new element, x, into a B*-tree of order m. How many disk accesses are 

made in the worst case and on the average? Assume that the B-tree was 

initially of depth / and that it 

is maintained on a disk. Each access retrieves or writes one node. 

Write an algorithm to delete the element whose key is x from a B*-tree of 

order m. What is the maximum number of accesses needed to delete from a 

B*-tree of depth !? Make the same assumptions as in Exercise 17. 


113 Bt-TREES 


113.1 Definition 


AB*-tree is a close cousin of the B-tree. The essential differences are: 


qd) 


Ina B*-tree we have two types of nodes—index and data. The index nodes 
of a BY-tree correspond to the internal nodes of a B-tree while the data 
nodes correspond to external nodes. The index nodes store keys (not ele- 
ments) and pointers and the data nodes store elements (together with their 
keys but no pointers). 

The data nodes are linked together, in left to right order, to form a doubly 
linked list. 


Figure 11.11 gives an example Bt-tree of order 3. The data nodes are 


shaded white the index nodes are not. Notice that the index nodes form a 2-3 tee 
whose height is 2. The capacity of a data node need not be the same as that of an 
index node. In Figure 11.11 each data node can hold 3 elements while each index 
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node can hold 2 keys. 
A 
20 | 40 
B Cc D 
10 ] 30 70 | 80 

2) fig o| 2 
4 jem 
6 48 EA 4 


Figure 11.11: A B”-tree of order 3 


Definition: A B*-tree of order mis a tree that either is empty or satisfies the fol- 

lowing properties: 

(1) All data nodes are at the same level and are leaves. Data nodes contain 
elements only. 

(2) The index nodes define a B-tree of order m; each index node has keys but 
not elements. 

(3) Let 


1, Ao, (Ky A) (KzAah ** "+ Bava) 


where the A;,0Si<n<m, are pointers to subtrees. and the 
K,, 1<i sn <m, are keys be the format of some index node. Let Kg = -9 
and K;,,; = 0°. All elements in the subtree A; have key less than X;4, and 


greater than or equal to K;,0Si Sn. 0 


11.3.2 Searching a B*-Tree 


Bt -trees support two types of searches— exact match and range. To search the 
tree of Figure 11.11 for the element whose key is pe we begin at the root A, 
de. From the definition of a B-tree we know that all ele- 


which is an index no kK 
ments in the left subtree of A (ie., the subtree whose root is B) have a key 
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smaller than 20; those in the subtree with root C have keys > 20 and < 40; and 
those in the subtree with root D have keys 2 40. So, the search moves to the 
index node C. Since the search key is > 30, the search moves from C to the data 
node that contains the elements with keys 32 and 36. This data node is searched 


and the desired element reported. Program 11.4 gives a high-level description of 
the algorithm to search a B*-tree, 


1 Search a B*-tree for an element with key x. 

// Return the element if found. Return NULL otherwise. 
if the tree is empty return NULL; 

Ko =— MAXKEY; 

for (*p = roor; p is an index node; p=Aj) 


Let p have the format n, Ag. (K1,A1), °*°. (Kas Andi 
K,,4; = MAXKEY; 
Determine i such that K; Sx < Kj41; 


} 

H search the data node p 

Search p for an element E with key x; 
if such an element is found return E 
else return NULL; 


aaa yg = CU 


Program 11.4: Searching a B-tree 


To search for all elements with keys in the range (16, 70], we proceed as in 
an exact match search for the start, 16, of the range. This gets us to the second 
data node in Figure 11.11. From here, we march down (rightward) the doubly 
linked list of data nodes until we reach a data node that has an element whose 
key exceeds the end, 70, of the search range (or until we reach the end of the 
list). In our example, 4 additional data nodes are examined. All examined data 
nodes other than the first and last contain at least one element that is in the 
search range. 


11.3.3 Insertion into a B*-Tree 


An important difference between inserting into a B-tree and inserting into a Bt. 
tree is how we handle the splitting of a data node. When a data node becomes 
overfull, half the elements (those with the largest keys) are moved into a new 
nude; the key of the smallest element so moved together with a pointer to the 
newly created data node are inserted into the parent index node (if any) using the 
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insertion procedure for a B-tree. The splitting of an index node is identical to the 
splitting of an internal node of a B-tree. 

Consider inserting an element with key 27 into the B*-tree of Figure 11.11. 

We first search for this key. The search gets us to the data node that is the left 
child of C. Since this data node contains no element with key 27 and since this 
data node isn’t full, we insert the new element as the third element in this data 
node. Next, consider the insertion of an element with key 14. The search for 14 
gets us to the second data node, which is full. Symbolically insening the new 
element into this full node results in an overfull node with the key sequence 12, 
14, 16, 18. The overfull node is split into two by moving the largest half of the 
elements (those with keys 16 and 18) into a new data node, which is then 
inserted into the doubly linked list of data nodes, The smallest key, 16, in this 
new data node together with a pointer to the new data node are inserted in the 
parent index node B to get the configuration of Figure 11.12 (a). 

Finally, consider inserting an element with key 86 into the B*-tee of Fig- 
ure 11.12 (a). The search for 86 gets us to the rightmost data node, which is full. 
Symbolically inserting the new element into this node results in the key 
sequence 80, 82, 84, 86. Splitting the overfull data node creates a new data node 
with the elements whose keys are 84 and 86. The new data node is inserted into 
the doubly linked list of data nodes. Then we insert the key 84 and a pointer to 
the new data node into the parent index node D, which becomes overfull. The 
overfull D is split using Eq. 11.5. The 84 along with two of the 4 subtrees of the 
overfull D are moved into a new index node E and the 80 together with a pointer 
to E inserted into the parent A of D. This causes A to become overfull. The over- 
full A is split using Eq. 11.5 and we get a new index node F that has the key 80 
and 2 of the 4 subtrees of the ov erfull A. The key 40 together with pointers to A 
and F form the new root of the B”-tree (Figure 11.12 (b)). 


11.3.4 Deletion from a B*-Tree 


Since elements are stored only in the leaves of a BY-tree, we need concem our- 
selves only with deletion from a leaf (recall that in the case of a B-tree we had to 
transform a deletion from a non-leaf into a deletion from a leaf; this case doesn’t 
arise for B"-trees). Since the index nodes of a B °-tree form a B-tree, a non-root 
index node is deficient when it has fewer than [nt/2] - | keys and a root index 
node is deficient when it has no key. When is a data node deficient? The 
definition of a B*-tree doesn't specify a minimum occupancy for a data node. 
However, we may get some guidance from our algorithm to insert an element, 
Following the split of an overfutl data node, the original data node as well as the 
new one each have at least [c/2] elements, where ¢ is the capacity of a data 
node. So, except when a data node is the root of the B"-tree, its occupancy is at 
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(b) 86 inserted 


Figure 11.12: Insertion into the B*-tree of Figure 11.11 


least [c/2]. We shall say that a non-root data node is deficient iff it has fewer 
than [c/2] elements; a root data node is deficient iff it is empty. 

We illustrate the deletion process by an example. Consider the Bt tree of 
Figure 11.11. The capacity ¢ of a data node is 3. So, a non-root data node is 
deficient iff it has fewer than 2 elements. To delete the element whose key is 40, 
we first search for the element to be deleted. This element is found in the data 
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node that is the left child of the index node D. Following the deletion of the ele- 
ment with key 40, the occupancy of this data node becomes 2. So, the data node 
isn’t deficient and we need merely write the modified data node to disk (assum- 
ing the B*-tree is disk resident) and we are done, Notice that when the deletion 
of an element doesn’t result in a deficient data node, no index node is changed. 

Next consider the deletion of the element whose key is 7] from the B-tree 
of Figure 11.11. This element is found in the middle child of D. Following its 
deletion, the middle child of D becomes deficient. We check either its nearest 
right or nearest left sibling and determine whether the checked sibling has more 
than the required minimum number ({¢/2}) of elements. Suppose we check the 
nearest right sibling, which has the key sequence 80, 82, 84. Since this node has 
an excess element, we borrow the smallest and update the in-between key in the 
parent D from 80 to that of the smallest remaining element in the right sibling, 
82. Figure 11.13 (a) shows the result. For a disk-resident B*-tree, this deletion 
would require us to write out one altered index node (D) and two altered data 
nodes. For data nodes with larger capacity, when a data node becomes deficient, 
we may borrow several elements from a nearest sibling that has excess elements. 
For example, when c=10, a deficient data node will have 4 elements and its 
nearest sibling may have 10. We could borrow 3 elements from the nearest 
sibling thereby balancing the occupancy in both data nodes to 7. Such a balanc- 
ing is expected to improve performance. ‘ 

When we delete the element with key 80 from the B”-tree of Figure 11.13 
{a), the middle child of D becomes deficient. We check its nearest right sibling 
and discover that this sibling has only [c/2} elements. So, the 2 data nodes are 
combined into one and the in-between key 82 that is in the parent index node D 
deleted. Figure 11.13 (b) shows the resulting B”-tree. Notice that combining two 
data nodes into one requires the deletion of a data node from the doubly linked 
list of data nodes. Note also that in the case of a disk resident B”-tree, the just 
performed deletion requires us to write out one altered data node (the middle 
child of D) and one altered index node (D). 

As another example for deletion, consider deleting the element with key 32 
from the B*-tree of Figure 11.12 (b). This element is in the middle child of C. 
Following the deletion, the middle child becomes deficient. Since its nearest 
sibling has only [c¢/2] elements, we cannot borrow from it. Instead, we combine 
the two data nodes deleting one from its doubly linked list and delete the in- 
between key (30) in the parent. Figure 11.14 (a) shows the result. As we can see, 
the index node C now has become deficient. When an index node becomes 
deficient, we examine a nearest sibling. If the examined nearest sibling has 
excess keys, we balance the occupancy of the two index nodes; this balancing 
involves moving some keys and associated subtrees as well as changing the in- 
between key in the parent. For our example, the in-between key 20 is moved 
from A to C, the rightmost key [6 of B is moved to A, and the right subtree of B - 
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(b) 80 deleted from (a) 


Figure 11.13: Deletion froma BY -tree 


moved to C. Figure !1.14 (b) shows the resulting BY -tree. 

As a final example, consider the deletion of the element with key 86 from 
the Bt-tree of Figure 11.12 (b). The middle child of E becomes deficient and is 
combined with its sibling; a data node is deleted from the doubly linked list of 
data nodes and the in-between key 84 in the parent also is deleted. This results in 
a deficient index node E and the configuration of Figure 11.15 (a). The deficient 
index node E combines with its sibling index node D and the in-between key 80 
to get the configuration of Figure 11.15 (b). Finally, the deficient index node F 
combines with its sibling A and the in-between key 40 in its parent G. This 
causes the parent G, which is the root, to become deficient. The deficient root is 
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(b) After borrowing from B 


Figure 11.14: Stages in deleting 32 from the B-tree of Figure 11.12 (b) 
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(a) E becomes deficient 


(b) F becomes deficient 


Figure 11.15: Stages in deleting 86 from the B”-tree of Figure 11.12 (b) 
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discarded and we get the Bt tree of Figure 11.12 (a). In the case of a disk 
resident B’-tree, the deletion of 86 would require us to write to disk one altered 
data node and 2 altered index nodes (A and D). 


EXERCISES 


1, 


Into the B*-tree of Figure 11.11 insert elements with keys 5, 38, 45. Jt and 
8 Ldn this order). Use the insertion method described in the text. Draw the 
B*-tree following each insert. 

Provide a high-level description (similar to Program 11.4) of the algorithm 
to insert into a B’-tree. 

Suppose that a BY -tree whose height is A is disk resident. How many disk 
accesses are needed, in the worst case, to insert a new elernent? Assume 
that each node may be read/written with a single access and that we have 
sufficient memory to save the h nodes accessed in the search phase so that 
these nodes don’t have to be re-read during the bottom-up node splitting 
phase. 

From the B*-tree of Figure 11.12 (b) delete the elements with keys 6, 71, 
14, 18, 16 and 2 (in this order). Use the deletion method described in the 
text. Show the B*-tree following cach delete. 

Provide a high-level description (similar to Program 11.4) of the algorithm 
to delete froma B’ -tree. 

Suppose that a B*-tree whose height is A is disk resident. How many disk 
accesses are needed, in the worst case, to delete an element? Assume that 
each node may be read/written with a single access and that we have 
sufficient memory to save the / nodes accessed in the search phase so that 
these nodes don’t have to be re-read during the bottom-up borrow and com- 
bine phase. 

Discuss the merits/demerits of replacing the doubly linked list of data 
nodes in a B’-tree by a singly linked list. 

Program the C++ class BPlusTree that implements a Bt tree. Your class 
must include functions for exact and range search as well as for insert and 
delete. Test all functions using your own test data. 


11.4 REFERENCES AND SELECTED READINGS 


B-trees were invented by Bayer and McCreight. For further reading on B-trees 
and their variants, see ‘Organization and maintenance of large ordered indices, 
by R. Bayer and E. McCreight, Acta Informatica, 1972; The art of computer 
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programming, Vol. 3, Sorting and Searching, Second Edition, by D. Knuth, 
Addison Wesley, 1997; “The ubiquitous B-tree,"” by D. Comer, ACM Computing 
Surveys, 1979; and ** B trees,” by D. Zhang, in Handbook of data structures and 
applications, D. Mehta and S. Sahni editors, Chapman & Hall/CRC, 2005. 
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Digital Search Structures 


12.1 DIGITAL SEARCH TREES 
12.1.1 Definition 


A digital search tree is a binary wee in which each node contains one element. 
The element-to-node assignment is determined by the binary representation of 
the element keys. Suppose that we number the bits in the binary representation 
of a key from left to right beginning at one. Then bit one of 1000 is 1, and bits 
two, three, and four are 0. All keys in the left subtree of a node at level i have bit 
# equal to zero whereas those in the right subtree of nodes at this level have bit i 
= I. Figure 12.1(a) shows a digital search tree. This tree contains the keys 1000, 


0010, 1001, 0001, 1100, and 0000. 
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Figure 12.1: Digital search trees 
12.1.2 Search, Insert and Delete 


Suppose we are to search for the key k = 0011 in the tree of Figure 12.1(a). k is 
first compared with the key in the root. Since & is different from the key in the 
root, and since bit one of k is 0, we move to the left child, b, of the root. Now, 
since k is different from the key in node b, and bit two of k is 0, we move to the 
left child, d, of b. Since & is different from the key in node d and since bit three 
of k is one, we move to the right child of d. Node d has no right child to move to. 
From this we conclude that k = 0011 is not in the search tree. If we wish to insert 
k into the tree, then it is to be added as the right child of d. When this is done, 
we get the digital search tree of Figure 12.1(b). 

The digital search tree functions to search and insert are quite similar to the 
corresponding functions for binary search trees. The essential difference is that 
the subtree to move to is determined by a bit in the search key rather than by the 
result of the comparison of the search key and the key in the current node. The 
deletion of an item in a leaf is done by removing the leaf node. To delete from 
any other node, the deleted item must be replaced by a value from any leaf in its 
subtrec and that leaf removed. 

Each of these operations can be performed in O(h) time, where Af is the 
height of the digital search tree. If each key in a digital search tree has keySize 
bits, then the height of the digital search tree is at most keySize + 1. 
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EXERCISES 
1. Draw a different digital search tree than Figure 12.1 (a) that has the same 
set of keys. 


2. Write the digital search tree functions for the search, insert, and delete 
operations. Assume that each key has keySize bits and that the function 
bit(k, i) returns the ith (from the left) bit of the key & Show that each of 
your functions has complexity O(h), where h is the height of the digital 


search tree. 


12.2 BINARY TRIES AND PATRICIA 


When we are dealing with very long keys, the cost of a key comparison is high. 
Since searching a digital search tree requires many comparisons between pairs of 
keys, digital search trees {and also binary and multiway search wees) are 
inefficient search structures when the keys are very long. We can reduce the 
number of key comparisons done during a search to one by using a related struc- 
ture called Patricia (Practical algorithm 10 retrieve information coded in 
alphanumeric). We shall develop this structure in three steps. First, we intro- 
duce a stricture called a binary trie. Then we transform binary tries into 
compressed binary tries, Finally, from compressed binary tries we obtain Patri- 
cia. Since binary tries and compressed binary tries are introduced only as a 
means of arriving at Patricia, we do not dwell much on how to manipulate these 
structures. A more general version of binary ties (called a trie) is considered in 
the next section. 


12.2.1 Binary Tries 


A binary trie is a binary tree that has two kinds of nodes: branch nodes and ele- 
ment nodes. A branch node has the two data members leftChild and rightChild. 
It has no data data member. An element node has the single data member data. 
Branch nodes are used to build a binary tree search structure similar to that of a 
digital search tree. This search structure leads to element nodes. 

Figure 12.2 shows a six-element binary trie. Element nodes are shaded. To 
search for an element with key , we use a branching pattern determined by the 
bits of k. The ith bit of & is used at level i. If it is zero, the search moves to the 
left subtree. Otherwise, it moves to the right subtree. To search for 0010 we first 
follow the left child, then again the left child, and finally the right child. 

Observe that a successful search in a binary trie always ends at an element 
node. Once this element node is reached, the key in this node ts compared with 
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Figure 12.2: Example of a binary trie 


the key we are searching for. This is the only key comparison that takes place. 
An unsuccessful search may terminate either at an element node or at a 0 pointer. 


12.2.2. Compressed Binary Tries 


The binary trie of Figure 12.2 contains branch nodes whose degree is one. By 
adding another data member, bitNumber, to each branch node, we can eliminate 
all degree-one branch nodes from the trie. The bitNumber data member of a 
branch node gives the bit number of the key that is to be used at this node. Fig- 
ure 12.3 gives the binary trie that results from the elimination of degree-one 
branch nodes from the binary trie of Figure 12.2. The number outside a node is 
its bitNumber. A binary wie that has been modified in this way to contain no 
branch nodes of degree one is called a compressed binary trie. 


12.2.3 Patricia 


Compressed binary tries may be represented using nodes of a single type. The 
new nodes, called augntented branch nodes, are the original branch nodes aug- 
mented by the data member cata. The resulting structure is called Patricia and 
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Figure 12.3: Binary trie of Figure 12.2 with degree-one nodes eliminated 


is obtained from a compressed binary trie in the following way: 


(ty 
(2) 
G) 


(4) 


Replace each branch node by an augmented branch node. 

Eliminate the element nodes. 

Store the data previously in the element nodes in the dara data members of 
the augmented branch nodes. Since every nonempty compressed binary 
trie has one less branch node than it has element nodes, it is necessary to 
add one augmented branch node. This node is called the header node. The 
temaining structure is the left subtree of the header node. The header node 
has bitNumber equal to zero. Its rightChild data member is not used. The 
assignment of data to augmented branch nodes is done in such a way that 
the bitNumber in the augmented branch node is less than or equal to that in 
the parent of the element node that contained this data. 

Replace the original pointers to element nodes by pointers to the respective 
augmented branch nodes. 


When these transformations are performed on the compressed trie of Figure 


12.3, we get the structure of Figure 12.4. Let root be the root of Patricia. root is 
O iff the Patricia is empty. A Patricia with one element is represented by a header 
node whose left-child data member points to itself (Figure 12.5(a)). We can dis- 
tinguish between pointers that pointed originally to branch nodes and those that 
pointed to element nodes by noting that, in Patricia, the former pointers are 
directed to nodes with a greater bitNwnber value, whereas pointers of the latter 
type are directed to nodes whose bitNumber value either is equal to or less than 
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Figure 12.4: An example of Patricia 


that in the node where the pointer originates. 


12.2.3.1 Searching Patricia 


To search for an element with key k, we begin at the header node and follow a 
path determined by the bits in k. When an element pointer is followed, the key in 
the node reached is compared with k. This is the only key comparison made. No 
comparisons are made on the way down. Suppose we wish to search for k = 
0000 in the Patricia instance of Figure 12.4. We begin at the header node and 
follow the left-child pointer to the node with 0000. The bit-number data member 
of this node is 1. Since bit one of k is 0, we follow the left child pointer to the 
node with 0010. Now bit three of k is used. Since this is 0, the search moves to 
the node with 0001. The bit-number data member of this node is 4. The fourth 
bit of & is zero, so we follow the left-child pointer. This brings us to a node with 
bit-number data member less than that of the node we moved from. Hence, an 
element pointer was used. Comparing the key in this node with k, we find a 
match, and the search is successful. 

Next, suppose that we are to search for k = 1011. We begin at the header 
node, The search moves successively to the nodes with 0000, 1001, 1000, and 
1001. & is compared with 1001. Since & is not equal to 1001, we conclude that 
there is no element with this key. The function to search a Patricia instance is 
given in Program 12.1. In this function, PatNode is the data type of the nodes in 
the wee and the function b/t (i,j) returns the jth bit (the leftmost bit is bit one) of 
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template <class X, class E> 

E* Patricia<K, E>::Search(const K& k) const 

{/ Search Patricia. Return a pointer to the element whose key is k. 

# Return NULL if no such element 
if (troot) return NULL; / Patricia is empty 
PatNode<K, E> *y = root-slefiChild; if move to left child of header node 
for (PatNode<K, E> *p = root; y—»bitNumber > p—»bitNumber;) 
{// follow a branch pointer 


p=y 
if (bit(k, y-sbitNumber)) y = y~rrightChilds 
else y = y—vleftChild; 


4 Check key in y 
if ( y -»key == k) return &y element, 
} return NULL; 


Program 12.1: Searching Patricia 


12.2.3.2 Inserting into Patricia 


Let us now examine how we can insert new elements. Suppose we begin with an 
empty instance and wish to insert an element with key 1000. The result is an 
instance that has only a header node (Figure 12.5(a)). Next, consider inserting 
an element with key & = 0010. First, we search for this key using function 
Patriciaz:Search (Program 12.1), The search terminates at the header node. 
Since 0010 is not equal to the key g = 1000 in this node, we know that 0010 is 
not currently in the Patricia instance, so the element may be inserted. For this, 
the keys k and g are compared to determine the first (Le. leftmost) bit at which 
they differ. This is bit one. A new node containing the element with key k is 
added as the left-child of the header node. Since bit one of & is zero, the left 
child data member of this new node poinls io itself, and its Fight-child data 
member points to the header node. Litany tha member is set to !. The 
resulting Patricia instance is shown in Figure 12.9(0). 

Suppose that the next element to be inserted has k = 1001. rea aa for 
this key ends at the node with g = 1000. The first bit at which k ale iffer be 
7 =4. Now we search the instance of Figure 12.5(b) using only the first j - 1 = 3 
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bits of k. The last move is from the node with 0010 to that with 1000. Since this 
is a right-child move, a new node containing the element with key k is to be 
inserted as the right child of 0010. The bit-number data member of this node is 
set to j = 4, As bit four of & is 1, the right-child data member of the new node 
points (o itself and its left-child data member points to the node with g. Figure 
12.5(c) shows the resulting structure. 

To insert k = 1100 into Figure 12.5(c), we first search for this key. Once 
again, q = 1000. The first bit at which k and q differ is j = 2. The search using 
only the first j — 1 bits ends at the node with 1001. The last move is a right child 
move from 0010. A new node containing the element with key & and bit-number 
data member j = 2 is added as the right child of 0010. Since bit j of k is one, the 
right-child data member of the new node points to itself. Its left-child data 
member points to the node with 1001 (this was previously the right child of 
0010). The new Patricia instance is shown in Figure 12.5(d). Figure 12.5(e) 
shows the result of inserting an element with key 0000, and Figure 12.5(f) shows 
the Patricia instance following the insertion of 0001. 

The preceding discussion leads to the insertion function Patricia::insert 
(Program 12.2). Its complexity is O(h), where A is the height of the Patricia. 
can be as large as min{keySize + 1, n}, where keySize is the number of bits in a 
key and n is the number of elements. When the keys are uniformly distributed, 
the height is O(!og n). We leave the development of the deletion function as an 
exercise. 


EXERCISES 


|. Write the binary trie functions for the search, insert, and delete operations. 
Assume that each key has keySize bits and that the function bit(k, i) 
returns the ith (from the left) bit of the key k. Show that each of your func- 
tions has complexity O(/), where h is the height of the binary trie. 

2. Write the compressed binary trie functions for the search, insert, and delete 
operations. Assume that each key has keySize bits and that the function 
bit (k, i) returns the ith (from the left) bit of the key k. Show that each of 
your functions has complexity O(#), where fA is the height of the 
compressed binary trie. 

3. Write a function to delete the element with key k from a Patricia. The com- 
plexity of your function should be O(/), where A is the height of the Patri- 
cia instance. Show that this is the case. 
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Figure 12.5: Insertion into Patricia 
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template <class K, class E> 
void Patricia<K, E>::Insert(const K& k, const E& ¢) 
{4 Insert ¢ into the Patricia tree. k is the key. 


} 


if (troor) { # Patricia is empty 
root = new PatNode<K, E>(0, k, ¢); 


4 Create a PatNode and set its bitNumber, key and element fields 


root—leftChild = root; return; 


} 
PatNode<K, E> *y = NSearch(k); 

H NSearch returns pointer to last node seen in search for k 
if (VWkey == K){ yelement = e; return;} 

4 Update old element 


# New element. A new node with ¢ is to be inserted 
Find first bit where k and y key differ 
for (int j = 1; bit(k, j) = bit(ykey, j); j++) 3 


Search Patricia using first j — I bits of k 
PatNode<K, E> *s = rootleftChild, *p = root; 
while ((s>bitNumber > pbitNumber) && (s—bitNumber < j)) { 
=55 
if (tlk, s—bitNumber))) s = sleftChild; 
else s = s—rightChild; 


Inset x as a child of p 

PatNode<K, E> *z = new PatNode<K, E> (j, k, €)} 
if (bit(k,j)) (z—9leftChild = s; 2-srightChild = 23} 
else {z->leftChild = z; zrightChild = s;} 


if (s == pleftChild) pleftChild = z; 
else p—rightChild = 2; 
return; 


Program 12.2: Insertion into Patricia 
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12.3. MULTIWAY TRIES 


12.3.1 Definition 


A multiway trie (or, simply, trie) is a structure that is particularly useful when 
key values are of varying size. This data structure is a generalization of the 
binary trie that was developed in the preceding section. 

A trie is a tree of degree m 2 2 in which the branching at any level is deter- 
mined not by the entire key value, but by only a portion of it. As an example, 
consider the trie of Figure 12.6 in which the keys are composed of lowercase 
letters from the English alphabet. The tie contains two types of nodes; element, 
and branch. In Figure 12.6, element nodes are shaded while branch nodes are 
not shaded. An element node has only a dara field; a branch node contains 
pointers to subtrees. In Figure 12.6, each branch node has 27 pointer fields. The 
extra pointer field is used for the blank character (denoted b). This character is 
used to terminate all keys, as a trie requires that no key be a prefix of another 
(see Figure 12.7). 


bluebird 


bunting 


cardinal 


thrasher 


Figure 12.6: Trie created using characters of key value from left to right, one at a time 


At the first level ail key values are partitioned into disjoint classes depend- 
ing on their first character. Thus, root—schild[i} points to 2 subtrie containing 
all key values beginning with the ith Setter. On the jth level the branching is 
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Figure 12.7; Trie showing need for a terminal character (in this case a blank) 


determined by the jth character. When a subtrie contains only one key value, it 
is replaced by a node of type element. This node contains the key value, together 
with other relevant information, such as the address of the record with this key 
value. 

As another example of a trie, suppose that we have a collection of student 
records that contain fields such as student name, major, date of birth, and social 
security number (SS#). The key field is the social security number, which is a 
nine digit decimal number. To keep the example manageable, assume that we 
have a total of five elements. Figure 12.8 shows the name and SS# fields for 
each of these five elements. 


Name SS# 

Jack 951-94-1654 
Sill 562-44-2169 
Bill 271-16-3624 
Kathy —278-49-1515 
April 951-23-7625 


Figure 12.8: Five elements (student records) 


To obtain a trie representation for these five elements, we first select a radix 
that wil} be used to decompose each key into digits. If we use the radix 10, the 
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decomposed digits are just the decimal digits shown in Figure 12.8. We shail 
examine the digits of the key field (i.c., SS#) from left to nght. Using the first 
digit of the SS#, we partition the elements into three groups—elements whose 
SS# begins with 2 (i.¢., Bill and Kathy), those that begin with 5 (i.e., Jill), and 
those that begin with 9 (i.e., April and Jack). Groups with more than one ele- 
ment are partitioned using the next digit in the key. This partitioning process is 
continued until every group has exactly one element in it, 

The partitioning process described above naturally results in a tree struc- 
ture that has 10-way branching as is shown in Figure 12.9, The tree employs two 
types of nodes—branch nodes and element nodes. Each branch node has 10 
children (or pointer) fields. These fields, child {0:9}, have been labeled 0,1, -+-.9 
for the root node of Figure 12.9. root.child{i] points to the root of a subtrie that 
contains all elements whose first digit is i. In Figure 12.9, nodes A,B,D,£,F, and 
J are branch nodes. The remaining nodes, nodes C,G,H_J, and K are element 
nodes. Each element node contains exactly one element, In Figure 12.9, only the 
key field of each element is shown in the element nodes. 


12.3.2 Searching a Trie 


Searching a trie for an element whose key is k requires breaking up & into its 
constituent characters/digits and following the branches determined by these 
characters. The function Trie::Search (Program 12.3) assumes that p = NULL is 
not a branch node and that the function digit (k.i) returns the ith digit of k. 


Analysis of Trie::Search: The search algorithm for ties is very straightforward, 
and one may readily verify that the worst-case search time is O(/), where / is the 
number of levels in the trie (including both branch and element nodes). , 
In the case of an index, all trie nodes will reside on disk, so at most J disk 
accesses will be made during search. When the nodes reside on disk, the CH 
pointer type cannot be used, as C++ does not allow inpuvoutput of pointers. The 


link data member will now be implemented as an integer. 7 


12.3.3. Sampling Strategies 


Given a set of key values to be represented in a0 index, the number of levels in 
the trie will depend on the strategy or key sampling technique used to determine 
the branching at each level. This can be Senne ty a ae finetions 

i i iately samples x for branching at the ith level. In the 
sample (x,i). which appropriately pee ae Senech (Programs 123)'this 


trie of Figure 12.6 and in the search i : t 
function ‘ sample (x.i) = ith character of.x. Some other choices for this function 


are 
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J | 95 1-23-7625 


Figure 12.9: Trie for the elements of Figure 12.8 


(1) sample (x,i) = Xn-ian 

(2) sample (x,i) = X,¢,,) for 7(x,i) a randomization function 
; .._ [tia ifi iseven 

(3) sample (x,i) = %y4i-tya if i is odd 
where x = x4X2 -"* X_- 

For each of these functions, one may easily construct key value sets for 
which the particular function is best (i.e., it results in a trie with the fewest 
number of levels). The trie of Figure 12.6 has five levels. Using function (1) on 
the same key values yields the trie of Figure 12.10, which has only three levels. 
An optimal sampling function for this data set will yield a trie that has only two 
levels (Figure 12.11). Choosing the optimal sampling function for any particular 
set of values is very difficult. In a dynamic situation, with insertion and deletion, 
we wish to optimize average performance. In the absence of any further 
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template <class K, E> 
E* Trie<K, E>::Search (const K& &) const 
{/ Search a trie, Return a pointer to the element whose key is k, 
4 Retum NULL if no such element 
TrieNode<K, E> *p = root; 
for(int i = 1; p is a branch node; i++) 
P = p—rchild (digit (k,i)]; 
if () == NULL Il pkey !=&) return NULL; 
else return &p— element; 
} 


Program 12.3: Searching a trie 


information on key values, the best choice would probably be function (2). 


bBabcdefghijkimnopqrstuvwxryz 


Figure 12.10: Trie constructed for data of Figure 12.6 sampling one character at a time, 
from right to left 


Although all our examples of sampling have involved single-character 
sampling we need not restrict ourselves to this. The key value may be inter: 
preted as consisting of digits using any radix we desire. Using a of 27 
would result in two-character sampling. Other radixes would give different sam- 
of levels in a trie can be kept low by adopting a 


plings. 
These nodes can be designed to hold more 


The maximum number 
different strategy for element nodes. 
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thrasher| | [cardinal Goshawk| | [wren [bunting godwit | 


| 
[chickadee| [bluebird] [gull] [oriole | [thrush | 


Figure 12.11: An optimal trie for the data of Figure 12.6 sampling on the first level done 
by using the fourth character of the key values 


than one key value. If the maximum number of levels allowed is /, then all key 
values that are synonyms up to level {- | are entered into the same element 
node. If the sampling function is chosen correctly, there will be only a few 
synonyms in each element node. The element node will therefore be small and 
can be processed in internal memory. Figure 12.12 shows the use of this strategy 
on the trie of Figure 12.6 with / = 3. In further discussion we shall, for simpli- 
city, assume that the sampling function in use is sample (x,i) = ith character of x 
and that no restriction is placed on the number of levels in the trie, 


bBabcdefghi 


CONT 


1 u 


I 
Lt 


bluebird || bunting 


cardinal | Fhickade 


Figure 12.12: Trie obtained for data of Figure 12.6 when number of levels is limited to 
3; keys have been sampled from left to right, one character at a time 
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12.3.4 Insertion into a Trie 


Insertion into a wie is straightforward. We j 
examples and leave the ali writing acs plete the procaine by two 
consider the trie of Figure 12.6 and insert into it the keys as op caeeclae, Let us 
First, we have x = bobwhite and we attempt to search for tabans ST luejay. 
us to node G, where we discover that o.ink[‘o'] = 0. He iy am I is eos 
and may be inserted here (see Figure 12.13). Next, x = bluejay ice te 
the trie leads us to the element node that contains bluebird. The ke a of 
and bluejay are sampled until the sampling results in two different th abi 
happens at the fifth letter. Figure 12.13 shows the trie after both Geeta: is 


p | bluebird 


Figure 12.13: Section of trie of Figure 12.6 showing changes resulting from inserting 
bobwhite and bluejay “ 


12.3.5 Deletion from a Trie 


Once again, we shall not present the deletion algorithm formally but will look at 
two examples to illustrate some of the ideas involved in deleting entries from a 
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trie. From the trie of Figure 12.13, let us first delete bobwhite. To do this we set 
o.link[‘o’] equal to 0, No other changes need to be made. Next, let us delete 
bluejay. This deletion leaves us with only one key value in the subtrie, 53. This 
means that the node 33 may be deleted, and p can be moved up one level. The 
same can be done for nodes 5, and 52. Finally, the node o is reached. The sub- 
trie with root o has more than one key value. Therefore, p cannot be moved up 
any more levels, and we set o.link[‘l’] equal to p. To facilitate deletions from 
tries, it is useful to add a count data member in each branch node. This data 
member contains the number of children the node has. 

As in the case of binary tries, we can define compressed tries in which each 
branch node has at least two children. In this case, each branch node is aug- 
mented to have an additional data member, skip, that indicates the number of 
levels of branching that have been eliminated (alternately, we can have a data 
member, sample, that indicates the sampling level to use). 


12.3.6 Keys With Different Length 


As noted earlier, the keys in a trie must be such that no key is a prefix of another. 
When alt keys are of the same length, as is the case in our SS# example of Figure 
12.9, this property is assured. But, when keys are of different length, as is the 
case with the keys of Figure 12.6, it is possible for one key to be a prefix of 
another. A popular way to handle such key collections is to append a special 
character such as a blank or a # that doesn’t appear in any key to the end of each 
key. This assures us that the modified keys (with the special character appended) 
satisfy the no-prefix property. 

An alternative to adding a special character at the end of each key is to 
give each node a dara field that is used to store the element (if any) whose key 
exhausts at that node. So, for example, the element whose key is 27 can be stored 
in node E of Figure 12.9. When this alternative is used, the search strategy is 
modified so that when the digits of the search key are exhausted, we examine the 
data field of the reached node. If this data field is empty, we have no element 
whose key equals the search key. Otherwise, the desired element is in this data 
field. 

It is important to note that in applications that have different length keys 
with the property that no key is a prefix of another, neither of the just-mentioned 
strategies is needed. 
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12.3.7 Height of a Trie 


Jn the worst case, a yoot-node to element-node has 
digit in a key. Therefore, the height of a tie is ee” every 
A trie for social security numbers has a height that is at most 10. If we 
assume that it takes the same time to move down one level of a tie as it does to 
move down one level of a binary search tree, then with at most 10 moves we can 
search a social-security trie. With this many moves, we can search a bin: 
search tree that has at most 2! ~ 1 = 1023 elements. This means that, we ex ae 
searches in the social security trie to be faster than searches in a binary Scerch 
tree (for student records) whenever the number of student records is more than 
1023. The breakeven point will actually be less than 1023 because we will nor- 
mally not be able to construct full or complete binary search trees for our ele- 
ment collection. 

Since a SS# is nine digits, a social security trie can have uy » 
in it. An AVL tree with 10° elements can have a height gh cleene 
(approximately) 1.4410g2(10°+2) = 44. Therefore, it could take us four times as 
much time to search for elements when we organize our collection of student 
records as an AVL wee rather than as a trie! 


42.3.8 Space Required and Alternative Node Structures 


The use of branch nodes that have as many child fields as the radix of the digits 
(or one more than this radix when different keys may have different length) 
results in a fast search algorithm. However, this node structure is often wasteful 
of space because many of the child fields are NULL. A radix r trie for d digit 
keys requires O(rdn) child fields, where n is the number of elements in the trie. 
To see this, notice that in a d digit trie with n clement nodes, each element node 
may have at most d ancestors, each of which is a branch node. Therefore, the 
number of branch nodes is at most dn. (Actually, we cannot have this many 
branch nodes, because the element nodes have common ancestors like the root 
node.) 

‘We can reduce the space requirements, at the expense of increased search 
time, by changing the node structure. Some of the possible alternative structures 
for the branch node of a trie are considered below. 


A chain of nodes, 
Each node of the chain has the three fields digitValue, child. and next. Node E of 


Figure 12.9, for example, would be replaced by the chain shown in Figure 12.14. 
The space required by a branch node changes from that required for r 
children/pointer fields to that required for 2p pointer fields and p digit value 
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firstNode —>{ 1 [s] + 81H un] 


Figure 12.14: Chain for node E of Figure 12.9 


fields, where p is the number of children fields in the branch node that are not 
NULL. Under the assumption that pointer fields and digit value fields are of the 
same size, a reduction in space is realized when more than two-thirds of the chil- 
dren fields in branch nodes are NULL. In the worst case, almost all the branch 
nodes have only | field that is not NULL and the space savings become almost 
(1-3/r)* 100%. 


A (balanced) binary search tree. 

Each node of the binary search tree has a digit value and a pointer to the subtrie 
for that digit value. Figure 12.15 shows the binary search tree for node E of Fig- 
ure 12.9, 


(8) 
© 


Figure 12.15: Binary search tree for node E of Figure 12.9 


Under the assumption that digit values and pointers take the same amount 
of space, the binary search tree representation requires space for 4p fields per 
branch node, because each search tree node has fields for a digit value, a subtrie 
pointer, a left child pointer, and a right child pointer. The binary search tree 
Tepresentation of a branch node saves us space when more than three-fourths of 
the children fields in branch nodes are NULL. Note that for large r, the binary 
search tree is faster to search than the chain described above. 


A binary trie, 
Figure 12.16 shows the binary trie for node E of Figure 12.9. The space required 
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by a branch node represented as a binary trie is at most (24 flogzr] +l)p. 


Figure 12.16: Binary trie for node E of Figure 12.9 


A hash table. 

When a hash table with a sufficiently small loading density is used, the expected 
time performance is about the same as when the node structure of Figure 12.9 is 
used. Since we expect the fraction of NULL child fields in a branch node to 
vary from node to node and also to increase as we go down the trie, maximum 
space efficiency is obtained by consolidating all of the branch nodes into a single 
hash table. To accomplish this, each node in the tie is assigned a number, and 
each parent to child pointer is replaced by a wiple of the form 
(currentNode ,digitValue,childNode). The numbering scheme for nodes is 
chosen so as to easily distinguish between branch and element nodes. For exam- 
ple, if we expect to have at most 100 elements in the trie at any time, the 
numbers 0 through 99 are reserved for element nodes and the numbers 100 on up 
are used for branch nodes. The element nodes are themselves represented as an 
array ¢lement {100}. (An alternative scheme is to represent pointers as tuples of 
the form (currentNode,digitValue,childNode,childNodelsBranchNode), where 
childNodelsBranchNode = true iff the child is a branch node.) 

Suppose that the nodes of the wie of Figure 12.9 are assigned numbers as 
given in Figure 12.17. This number assignment assumes that the trie will have 
no more than 10 elements. 

The pointers in node A are represented by the tuples (10,2,11),(10,5,0), 
and (10,9, 12). The pointers in node E are represented by the tuples (13.1.1) and 

13,8,2). 
; The pointer triples are stored in a hash table using the first two fields (i.e, 
the currentNode and digitValue) as the key. For this purpose, we may transform 
the two field key into an integer using the formula currentNode *r+digitValue, 
where r is the trie radix, and use the division method to hash the transformed key 
into a home bucket. The data presently in element node iis stored in element {i}. 


658 Digital Search Structures 


Node A BoC D E F G oH t 
Number 10 II Oo 12 13° 14 I 24 


a 
re 
AK 


Figure 12.17: Number assignment to nodes of trie of Figure 12.9 


To see how all this works, suppose we have set up the trie of Figure 12.9 
using the hash table scheme just described. Consider searching for an element 
with key 278-49-1515. We begin with the knowledge that the root node is 
assigned the number 10. Since the first digit of the search key is 2, we query our 
hash table for a pointer triple with key (10,2). The hash table search is successful 
and the triple (10,2,11) is retrieved. The childNode component of this triple is 
11, and since all element nodes have a number 9 or less, the child node is deter- 
mined to be a branch node. We make a move to the branch node 11. To move to 
the next level of the trie, we use the second digit 7 of the search key. For the 
move, we query the hash table for a pointer with key (11,7). Once again, the 
search is successful and the triple (11,7,13) is retrieved. The next query to the 
hash table is for a tiple with key (13,8). This time, we obtain the triple (13,8,2). 
Since, childNode = 2 < 10, we know that the pointer gets us to an element node. 
So, we compare the search key with the key of element [2]. The keys match, and 
we have found the element we were looking for. 

When searching for an element with key 322-16-8976, the first query is for 
a triple with key (10,3). The hash table has no triple with this key, and we con- 
clude that the trie has no element whose key equals the search key. 

The space needed for each pointer triple is about the same as that needed 
for each node in the chain of nodes representation of a trie node. Therefore, if we 
use a linear open addressed hash table with a loading density of a, the hash table 
scheme will take approximately (1/a-1)*100% more space than required by the 
chain of nodes scheme. However, when the hash table scheme is used, we can 
retrieve a pointer in O(1) expected time, whereas the time to retrieve a pointer 
using the chain of nodes scheme is O(). When the (balanced) binary search tree 
or binary trie schemes are used, it takes O(logr) time to retrieve a pointer. For 
large radixes, the hash table scheme provides significant space saving over the 
scheme of Figure 12.9 and results in a small constant factor degradation in the 
expected time required to perform a search. 

The hash table scheme actually reduces the expected time to insert ele- 
ments into a trie, because when the node structure of Figure 12.9 is used, we 
must spend O(r) time to initialize each new branch node {see the description of 
the insert operation below). However, when a hash table is used, the insertion 
time is independent of the trie radix. 
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‘To support the removal of elements from a trie represented as a hash table, 
we must be able to reuse element nodes. This reuse is accomplished by setting 
up an available space list of element nodes that are currently not in use. 


12.3.9 Prefix Search and Applications 


You have probably realized that to search a trie we do not need the entire key. 
Most of the time, only the first few digits (i.c., a prefix) of the key is needed. For 
example, our search of the tie of Figure 12.9 for an element with key 
951-23-7625 used only the first four digits of the key. The ability to search a wie 
using only the prefix of a key enables us to use tries in applications where only 
the prefix might be known or where we might desire the user to provide only a 
prefix. Some of these applications are described below. 


Criminology: Suppose that you are at the scene of a crime and observe the first 
few characters CRX on the registration plate of the getaway car. If we have a trie 
of registration numbers, we can use the characters CRX to reach a subtrie that 
contains all registration numbers that begin with CRX. The elements in this sub- 
trie can then be examined to see which cars satisfy other properties that might 
have been observed. 


Automatic Command Completion: When using an operating system such as 
Unix or Windows (command prompt), we type in system commands to accom- 
plish certain tasks. For example, the Unix and DOS command cd may be used to 
change the current directory. Figure 12.18 gives a list of commands that have 
the prefix ps (this list was obtained by executing the command /s 
/asr/tocal/bin/ps* on a Unix system). 


ps2ascii ps2pdf —_ psbook psmandup _psselect 
ps2epsi ps2pk pscal psmerge pstopnm 
ps2frag ps2ps psidtopgm — psnup pstops 
ps2gif psbb psiatex psresize pstruct 


Figure 12.18: Commands that begin with "ps" 


We can simplify the task of typing in commands by providing a command 
completion facility which automatically types in the command suffix once the 
user has typed in a long enough prefix to uniquely identify the command. For 
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instance, once the letters psi have been entered, we know that the command must 
be psidiopgm because there is only one command that has the prefix psi. In this 
case, we replace the need to type in a 9 character command name by the need to 
type in just the first 3 characters of the command! 

A command completion system is easily implemented when the commands 
are stored in a trie using ASCII characters as the digits. As the user types the 
command digits from left to right, we move down the trie. The command may be 
completed as soon as we reach an element node. If we fall off the trie in the pro- 
cess, the user can be informed that no command with the typed prefix exists. 

Although we have described command completion in the context of operat- 
ing system commands, the facilty is useful in other environments: 


(1) A web browser keeps a history of the URLs of sites that you have visited. 
By organizing this history as a trie, the user need only type the prefix of a 
previously used URL and the browser can complete the URL. 


(2) A word processor can maintain a collection of words and can complete 
words as you type the text. Words can be completed as soon as you have 
typed a long enough prefix to identify the word uniquely. 

(3) An automatic phone dialler can maintain a list of frequently called tele- 
phone numbers as a trie. Once you have punched in a long enough prefix to 
uniquely identify the phone number, the dialler can complete the call for 
you. 


12.3.10 Compressed Tries 


‘Take a close look at the trie of Figure 12.9. This trie has a few branch nodes 
(nodes B,D, and F) that do not partition the elements in their subtrie into two or 
more nonempty groups. We often can improve both the time and space perfor- 
mance metrics of a trie by eliminating all branch nodes that have only one child. 
The resulting trie is called a compressed trie. 

When branch nodes with a single child are removed from a trie, we need to 
keep additional information so that trie operations may be performed correctly. 
The udditional information stored in three compressed trie structures is described 
below. 
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42.3.10.1 Compressed Tries with Digit Numbers 


In a compressed trie with digit numbers, each branch node has an additional field 
digitNumber that tells us which digit of the key is used to branch at this node. 
Figure 12.19 shows the compressed trie with digit numbers that corresponds to 
the trie of Figure 12.9. The leftmost field of each branch node of Figure 12.19 is 
the digitNumber field. 


A 0123456789 


K 


278-49-1515 


G J 


Figure 12.19: Compressed trie with digit numbers 


12,3.10.2 Searching a Compressed Trie with Digit Numbers 


A compressed trie with digit numbers may be searched by following a path from 
the root. At each branch node, the digit, of the search key, given in the branch 
node’s digitNumber field is used to determine which subtrie to move to. For 
example, when searching the trie of Figure 12.19 for an element with key 951- 
23-7625, we start at the root of the trie. Since the root node is a branch node with 
digitNumber =1, we use the first digit 9 of the search key to determine which 
subtrie to move to. A move to node A.child[9}=/ is made. Since, 
LdigitNumber=4, the fourth digit, 2, of the search key tells us which subtrie to 
move to. A move is now made to node I.child[2]=J. We are now at an element 
node, and the search key is compared with the key of the element in node J. 
Since the keys match, we have found the desired element. 

Notice that a search for an element with key 913-23-7625 also terminates 
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at node J. However, the search key and the element key at node J do not match 
and we conclude that the trie contains no element with key 913-23-7625. 


12.3.10.3 Inserting into a Compressed Trie with Digit Numbers 


To insert an element with key 987-26-1615 into the trie of Figure 12.19, we first 
search for an element with this key. The search ends at node J. Since, the search 
key and the key, 95 1-23-7625, of the element in this node do not match, we con- 
clude that the trie has no element whose key matches the search key. To insert 
the new element, we find the first digit where the search key differs from the key 
in node J and create a branch node for this digit. Since, the first digit where the 
search key 987-26-1615 and the element key 951-23-7625 differ is the second 
digit, we create a branch node with digitNumber =2. Since, digit values increase 
as we go down the trie, the proper place to insert the new branch node can be 
determined by retracing the path from the root to node J and stopping as soon as 
either a node with digit value greater than 2 or the node J is reached. In the trie 
of Figure 12.19, this path retracing stops at node /. The new branch node is made 
the parent of node /, and we get the trie of Figure 12.20. 

Consider inserting an element with key 958-36-4194 into the compressed 
trie of Figure 12.19. The search for an element with this key terminates when we 
fall of the trie by following the pointer L.child{3]=NULL. To complete the inser- 
tion, we must first find an element in the subtrie rooted at node J. This element is 
found by following a downward path from node / using (say) the first non NULL 
link in each branch node encountered. Doing this on the compressed trie of Fig- 
urc 12.19, leads us to node J, Having reached an element node, we find the first 
digit where the element key and the search key differ and complete the insertion 
as in the previous example. Figure 12.21 shows the resulting compressed trie. 

Because of the possible need to search for the first non NULL child pointer 
in each branch node, the time required to insert an element into a compressed 
tries with digit numbers is O(rd), where r is the trie radix and d is the maximum 
number of digits in any key. 


12.3.10-4 Deleting an Element from a Compressed Trie with Digit 
Numbers 


To delete an element whose key is k, we do the following: 


(t) Find the element node X that contains the element whose key is k. 
(2) Discard node X. 
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Figure 12.20: Compressed trie following the insertion of 987-26-1615 into the 
compressed trie of Figure 12.19 


(3) If the parent of X is left with only one child, discard the parent node also. 
When the parent of X is discarded, the sole remaining child of the parent of 
X becomes a child of the grandparent (if any) of X. 


To remove the element with key 951-94-1654 from the compressed trie of 
Figure 12.21, we first locate the node K that contains the element that is to be 
removed. When this node is discarded, the parent / of K is left with only one 
child. Consequently, node / is also discarded, and the only remaining child J of 
node / is the made a child of the grandparent of K. Figure 12.22 shows the 
resulting compressed trie. 

Because of the need to determine whether a branch node is left with two or 
more children, removing a d digit element from a radix r trie takes O(d +r) time. 
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A 0123456789 
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958-36-4194 


271-16-3624 | | 278-49-1515 


“ 


Figure 12.21: Compressed trie following the insertion of 958-36-4194 into the 
compressed trie of Figure 12.19 


123.11 Compressed Tries with Skip Fields 


In a compressed trie with skip fields, each branch node has an additional field 
skip which tells us the number of branch nodes that were originally between the 
current branch node and its parent. Figure 12.23 shows the compressed trie with 
skip fields that corresponds to the trie of Figure 12.9. The leftmost field of each 
branch node of Figure 12.23 is the skip field. 

The algorithms to search, insert, and remove are very similar to those used 
for a compressed trie with digit numbers. 


12.3.12 Compressed Tries with Labeled Edges 


In a compressed trie with labeled edges. each branch node has the following 
additional intormation associated with it: a pointer/reference element to an ele- 
ment (or element node) in the subtrie, and an integer skip which equals the 
number of branch nodes eliminated between this branch node and its parent. 
Figure 12.24 shows the compressed wie with labeled edges that corresponds to. 
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271-16-3624]| [278-49-1515] 


Figure 12.22: Compressed trie following the removal of 951-94-1654 from the 
compressed trie of Figure 12.21 


L 
271-16-3624| | 278-49-1515] [951-23-7625] [95-94-1654] 


Figure 12.23: Compressed trie with skip fields 


the trie of Figure 12.9. The first field of each branch node is its element field, and 


the second field is the skip field. 
Even though we store the “‘fabel’’ with branch nodes, it is convenient to 


think of this information as being associated with the edge that comes into the 
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G 
951-94-1654 


H J 
271=16-3624| | 278-49-1515 951-23-7625 


Figure 12.24: Compressed trie with labeled edges 


branch node from its parent (when the branch node is not the root), When mov- 
ing down a trie, we follow edges, and when an edge is followed. we skip over the 
number of digits given by the skip field of the edge information. The value of the 
digits that are skipped over may be determined by using the element field. 

When moving from node A to node / of the compressed trie of Figure 
12.24, we use digit 1 of the key to determine which child field of A is to be used. 
Also, we skip over the next 2 digits, that is, digits 2 and 3, of the keys of the ele- 
ments in the subtrie rooted at 7. Since all elements in the subtrie J have the same 
value for the digits that are skipped over, we can determine the value of these 
skipped over digits from any of the elements in the subtrie. Using the element 
field of the edge label, we access the element node J, and determine that the 
digits that are skipped over are 5 and 1. 


12.3.12.1 Searching a Compressed Trie with Labeled Edges 


When searching a compressed trie with labeled edges, we can use the edge label 
to terminate unsuccessful searches (possibly) before we reach an element node 
or fall off the trie. As in the other compressed trie variants, the search is done by 
following a path from the root. Suppose we are searching the compressed trie of. 
Figure 12.24 for an element with key 921-23-1234. Since the skip value for the 
root node is 0, we use the first digit 9 of the search key to determine which sub- 
tne to move to. A move to node A.child (9]=/ is made. By examining the edge 
label (stored in node /), we determine that, in making the move from node A to 
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node /, the digits 5 and 1 are skipped. Since these digits do not agree with the 
next two digits of the search key, the search terminates with the conclusion that 
the trie contains no element whose key equals the search key. 


12.3.12.2 Inserting into a Compressed Trie with Labeled Edges 


To insert an element with key 987-26-1615 into the compressed trie of Figure 
12.24, we first search for an element with this key. The search terminates unsuc- 
cessfully when we move from node A to node / because of a mismatch between 
the skipped over digits and the corresponding digits of the search key. The first 
mismatch is at the first skipped over digit. Therefore, we insert a branch node L 
between nodes A and /. The skip value for this branch node is 0, and its element 
field is set to reference the element node for the newly inserted element. We 
must also change the skip value of / to 1. Figure 12.25 shows the resulting 


compressed trie. 


,987-26-1615 | 


[271-16-3624] [278-49-1515] Hi 


\ 
J k_\ 
951-94-1654] 


Figure 12.25: Compressed trie following the insertion of 987-26-1615 into the 
compressed trie of Figure 12.24 


Suppose we are to insert an element with key 958-36-4194 into the 
compressed trie of Figure 12.25. The search for an element with this key 
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terminates when we move to node / because of a mismatch between the digits 
that are skipped over and the corresponding digits of the search key. A new 
branch node is inserted between nodes A and / and we get the compressed trie 
that is shown in Figure 12.26. 


A__0123456789 


16 L 


Eo 8 
Gr {lt mahi [562-44-2169] 1 
ee 
of H 19 M 
271-16-3624 J 958-36-4194 
J K 


951-23-7625| | 951-94-1654 


Figure 12.26: Compressed trie following the insertion of 958-36-4194 into the 
compressed trie of Figure 12.24 


The time required to insert a d digit element into a radix r compressed trie 
with labeled edges is O(r +d). 


12.3.12.3 Deleting an Element from a Compressed Trie with Labeled 
Edges 


This is similar to removal from a compressed trie with digit numbers except for 
the need to update the element fields of branch nodes whose element field refer- 
ences the removed element. 
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12,3.13 Space Required by a Compressed Trie 


Since each branch node partitions the elements in its subtrie into two or more 
nonempty groups, an element compressed trie has at most n—1 branch nodes. 
Therefore, the space required by each of the compressed trie variants described 
by us is O(nr), where r is the trie radix. 

When compressed tries are represented as hash tables, we need an addi- 
tional data structure to store the nonpointer fields of branch nodes, We may use 
an array for this purpose. 


EXERCISES 
1. (a) Draw the trie obtained for the following data: 


AMIOT, AVENGER, AVRO, HEINKEL, HELLDIVER, MACCHI, 
MARAUDER, MUSTANG, SPITFIRE, SYKHOI 


Sample the keys from left to right one character at a time. 
(b) Using single-character sampling, obtain a tie with the fewest 
number of levels. 

2. Explain how a trie could be used to implement a spelling checker. 

3. Explain how a trie could be used to implement an auto-command comple- 
tion program. Such a program would maintain a library of valid com- 
mands. It would then accept a user command, character by character, from 
a keyboard. When a sufficient number of characters had been input to 
uniquely identify the command, it would display the complete command on 
the computer monitor. 

4, Write an algorithm to insert a key value x into a trie in which the keys are 
sampled from left to right, one character at a time, 

5. Do Exercise 4 with the added assumption that the trie is to have no more 
than six levels. Synonyms are to be packed into the same element node. 

6. Write an algorithm to delete x from a wie under the assumptions of Exer- 
cise 4. Assume that each branch node has a count data member equal to 
the number of element nodes in the subtrie for which it is the root. 


7. Do Exercise 6 for the trie of Exercise 5. 

8. In the wie of Figure 12.13 the nodes 5, and 8, each have only one child. 
Branch nodes with only one child may be eliminated from tries by main- 
taining a skip data member with each node. The value of this data member 
equals the number of characters to be skipped before obtaining the next 
character to be sampled. Thus, we can have skip [5;}=2 and delete the 
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nodes 5, and 5). Write algorithms to search, insert, and delete from tries in 
which each branch node has a skip data member. 


9. Assume that the branch nodes of a compressed trie are represented using a 
hash table (one for each node). Each such hash table is augmented with a 
count and skip value as described above. Describe how this change to the 
node structure affects the time and space complexity of the trie data struc- 
ture. 


10. Do the previous exercise for the case when each branch node is represented 
by a chain in which each node has two data members: pointer and link, 
where pointer points to a subtrie and link points to the next node in the 
chain, The number of nodes in the chain for any branch node equals the 
number of non-0 pointers in that node. Each chain is augmented by a skip 
value. Draw the chain representation of the compressed version of the trie 
of Figure 12.6. 


12.4 SUFFIX TREES 
12.4.1 Have You Seen This String? 


In the classical substring search problem, we are given a string S and a pattem P 
and are to report whether or not the pattern P occurs in the string S. For example, 
the pattern P = cat appears (twice) in the string S1 = The big cat ate the small 
catfish., but does not appear in the string $2 = Dogs for sale.. 

Researchers in the human genome project, for example, are constantly 
searching for substrings/patterns (we use the terms substring and pattern inter- 
changeably) in a gene databank that contains tens of thousands of genes. Each 
gene is represented as a sequence or string of letters drawn from the alphabet 
A,C,G,T. Although most of the strings in the databank are around 2000 letters 
long, some have tens of thousands of letters. Because of the size of the gene 
databank and the frequency with which substring searches are done, it is impera- 
tive that we have as fast an algorithm as possible to locate a given substring 
within the strings in the databank. 

We can search for a pattern P in a string S$ using Program 2.16. The com- 
plexity of such a search is O(IP t+1S1), where 1P1 denotes the length (i.c., 
number of letters or digits) of P. This complexity looks pretty good when you 
consider that the pattern P could appear anywhere in the string S. Therefore, we 
thust examine every letter/digit (we use the terms letter and digit interchange- 
ably) of the string before we can conclude that the search pattern does not appear 
in the string. Further, before we can conclude that the search pattern appears in 
the string, we must examine every digit of the pattern. Hence, every pattem 
seurch algorithm must take time that is linear in the lengths of the pattern and the 
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string being searched. 
When classical pattern matching algorithms are used to search for several 


patterns P;, P2, -**, Py in the string S, O(1P)1 + 1P2t+ -+- + 1Pyl + ISI) 
time is taken (because O(1P;1+1S 1) time is taken to seach for P;). The suffix 
tree data structure that we are about to study reduces this complexity to 
OUP t+ IP2l + ++: + 1Pyl +151). Of this time, O(1S1) time is spent set- 
ting up the suffix tree for the string S; an individual pattem search takes only 
OCP; 1) time (after the suffix tree for S has been built), Therefore once the suffix 
tree for S has been created, the time needed to search for a pattern depends only 


on the length of the pattern. 


12.4.2 The Suffix Tree Data Structure 


The suffix tree for S is actually the compressed wie for the nonempty suffixes of 
the string S. Since a suffix tree is a compressed trie, we sometimes refer to the 
tree as a trie and to its subtrees as subtries. 

The (nonempty) suffixes of the string S = peeper are peeper, eeper, eper, 
per, er, and r. Therefore, the suffix tree for the string peeper is the compressed 
trie that contains the elements (which are also the keys) peeper, eeper, eper, per, 
er, and r. The alphabet for the string peeper is e,p,r. Therefore, the radix of the 
compressed trie is 3. If necessary, we may use the mapping e +0, p 1, 
r — 2, to convert from the letters of the string to numbers. This conversion is 
necessary only when we use a node structure in which each node has an array of 
child pointers. Figure 12.27 shows the compressed trie (with labeled edges) for 
the suffixes of peeper. This compressed tie is also the suffix tree for the string 
peeper. 
Since the data in the element nodes D-I are the suffixes of peeper, each 
element node need retain only the start index of the suffix it contains. When the 
letters in peeper are indexed from left to right beginning with the index }, the 
element nodes D-/ need only retain the indexes 6, 2, 3, 5, 1, and 4, respectively. 
Using the index stored in an element node, we can access the suffix from the 
string S. Figure 12.28 shows the suffix tree of Figure 12.27 with each element 
node containing a suffix index. 

The first component of each branch node is a reference to an element in 
that subtrie. We may replace the element reference by the index of the first digit 
of the referenced element. Figure 12.29 shows the resulting compressed trie. We 
shall use this modified form as the representation for the suffix tree. 

When describing the search and construction algorithms for suffix trees, it 
is easier to deal with a drawing of the suffix tree in which the edges are labeled 
by the digits used in the move from a branch node to a child node. The first digit 
of the label is the digit used to determine which child is moved to, and the 
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Figure 12.27: Compressed trie for the suffixes of peeper 


Temaining digits of the label give the digits that are skipped over. Figure 12.30 
shows the suffix tree of Figure 12.29 drawn in this manner. 


Figure 12.30 A more humane drawing of a suffix tree 


In the more humane drawing of a suffix tree, the labels on the edges on any 
root to element node path spell out the suffix represented by that element node. 
When the digit number for the root is not 1, the humane drawing of a suffix tree 
includes a header node with an edge to the former root. This edge is labeled with 
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Figure 12.28: Modified compressed trie for the suffixes of peeper 


the digits that are skipped over. 

The string represented by a node of a suffix tree is the string formed by the 
labels on the path from the root to that node. Node A of Figure 12.30 represents 
the empty string €, node C represents the string pe, and node F represents the 
string eper. 

Since the keys in a suffix tree are of different length, we must ensure that no 
key is a proper prefix of another. Whenever the last digit of string $ appears only 
once in S, no suffix of $ can be a proper prefix of another suffix of §. In the string 
peeper, the last digit is r, and this digit appears only once. Therefore, no suffix of 
peeper is a proper prefix of another. The last digit of data is a, and this last digit 
appears twice in data. Therefore, dara has two suffixes ata and a that begin with 
a. The suffix a is a proper prefix of the suffix ata. 

When the last digit of the string $ appears more than once in S we must 
append a new digit (say #) to the suffixes of S so that no suffix is a prefix of 
another. Optionally, we may append the new digit to S to get the string S#, and 
then construct the suffix tree for S#. When this optional route is taken, the suffix 
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456 
s=[plelejplel*] 


Figure 12.29: Suffix tree for peeper 


tree has one more suffix (#) than the suffix tree obtained by appending the symbol 
# to the suffixes of S. 


12.4.3 Let’s Find That Substring (Searching a Suffix Tree) 


But First, Some Terminology 

Let n=151 denote the length (i.e., number of digits) of the string whose suffix 
tree we ure to build. We number the digits of S from left to right beginning with 
the number 1. S[{/] denotes the ith digit of S, and suffix(i) denotes the suffix 
S{é]--- S|] that begins at digit i, 1sisn. 


On With the Search 

A fundamental observation used when searching for a pattern P in a string S is 

that P appears in S (i.e., P is a substring of S) iff P is a prefix of some suffix of S. 
Suppose that P=P[1] --- P(A] =S[i] --- S(i+k-2]. Then, P is a prefix 
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of suffix(i). Since sufix (i) is in our compressed trie (.e., suffix tree), we can 
search for P by using the strategy to search for a key prefix ina compressed trie, 

Let’s search for the pattern P = per in the string S$ = peeper. Imagine that 
we have already constructed the suffix tree (Figure 12,30) for peeper. The search 
starts at the root node A. Since P(1] =p, we follow the edge whose label begins 
with the digit p. When following this edge, we compare the remaining digits of 
the edge label with successive digits of P. Since these remaining label digits 
agree with the pattern digits, we reach the branch node C. In getting to node C, 
we have used the first two digits of the pattern. The third digit of the pattem is r, 
and so, from node C we follow the edge whose label begins with r. Since this 
edge has no additional digits in its label, no additional digit comparisons are 
done and we reach the element node J. At this time, the digits in the pattern have 
been exhausted and we conclude that the pattern is in the string. Since an ele- 
ment node is reached, we conclude that the pattern is actually a suffix of the 
string peeper. In the actual suffix tree representation (rather than in the humane 
drawing), the element node / contains the index 4 which tells us that the pattem 
P = per begins at digit 4 of peeper (i.e., P = suffix(4)). Further, we can conclude 
that per appears exactly once in peeper; the search for a pattern that appears 
more than once terminates at a branch node, not at an element node. 

Now, tet us search for the pattern P = eeee. Again, we start at the root. 
Since the first character of the pattern is ¢, we follow the edge whose label 
begins with e and reach the node B. The next digit of the pattern is also e, and 
so, from node B we follow the edge whose label begins with e. In following this 
edge, we must compare the remaining digits per of the edge label with the fol- 
lowing digits ee of the pattern. We find a mismatch when the first pair (p.e) of 
digits are compared and we conclude that the pattern does not appear in peeper. 

Suppose we are to search for the pattem P =p. From the root, we follow 
the edge whose label begins with p. In following this edge, we compare the 
remaining digits (only the digit e remains) of the edge label with the following 
digits (there aren’t any) of the pattern. Since the pattern is exhausted while fol- 
lowing this edge, we conclude that the pattern is a prefix of all keys in the subtrie 
rooted at node C. We can find all occurrences of the pattern by traversing the 
subtrie rooted at C and visiting the information nodes in this subtrie. If we want 
the location of just one of the occurrences of the pattern, we can use the index 
stored in the first component of the branch node C (see Figure 12.29). When a 
pattern exhausts while following the edge to node X, we say that node X has been 
reached; the search terminates at node X. 

When searching for the pattern P = rope, we use the first digit r of P and 
reach the element node D. Since the the pattern has not been exhausted, we must 
check the remaining digits of the pattern against those of the key in D. This 
check reveals that the pattern is not a prefix of the key in D, and so the pattem 
does not appear in peeper. 
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The last search we are going to do is for the pattern P = pepe. Starting at 
the root of Figure 12,30, we move over the edge whose label begins with p and 
reach node C. The next unexamined digit of the search pattern is p. So, from 
node C. we wish to follow the edge whose label begins with p. Since no edge 
satisfies this requirement, we conclude that pepe does not appear in the string 
peeper. 


12.4.4 Other Nifty Things You Can Do with a Suffix Tree 


Once we have set up the suffix tree for a string S, we can tell whether or not S 
contains a pattern P in O{(P |) time. This means that if we have a suffix tree for 
the text of Shakespeare's play ‘‘Romeo and Juliet,”” we can determine whether or 
not the phrase wherefore art thou appears in this play with lightning speed. In 
fact, the time taken will be that needed to compare up to 18 (the length of the 
search pattern) letters/digits. The search time is independent of the length of the 
play 

Some other interesting things you can do at lightning speed are described 
below. 


Find all occurrences of a pattern P. 

This is done by searching the suffix tree for P. If P appears at least once, the 
search terminates successfully either at an element node or at a branch node. 
When the search terminates at an element node, the pattern occurs exactly once. 
When we terminate at a branch node X, all places where the pattern occurs can 
be found by visiting the element nodes in the subtrie rooted at X. This visiting 
can be done in time linear in the number of occurrences of the pattern if we 


(a) Link all of the element nodes in the suffix tree into a chain, the linking is 
done in lexicographic order of the represented suffixes (which also is the 
order in which the clement nodes are encountered in a left to right scan of 
the element nodes). ‘The element nodes of Figure 12.30 will be linked in the 
order E,F,G,H,1,D. 


{b) In each branch node, keep a reference to the first and last element node in 
the subtrie of which that branch node is the root. In Figure 12.30, nodes A, 
B, and C keep the pairs (£.D), (E,G), and (H,/), respectively. We use the 
pair (firsdnformationNode,lastinformationNode) to traverse the element 
node chain starting at firsténformationNode and ending at fastinformation- 
Node. This traversal yields all occurrences of patterns that begin with the 
suring spelled by the edge labels from the root to the branch node. Notice 
that when (firstinformationNode,lastinformationNode) pairs are kept in 
tianch nodes, we can eliminate the branch node field that keeps a reference 
tv un element node tn the subtrie (i.e. the field element). 
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Find all strings that contain a pattern P. 

Suppose we have a collection $1, $2, ---, Sk of strings and we wish to Teport 
all strings that contain a query pattern P. For example, the genome databank con- 
tains tens of thousands of strings, and when a researcher submits a query string, 
we are to report alJ databank strings that contain the query string. To answer 
queries of this type efficiently, we set up a compressed trie (we may call this a 
multiple string suffix tree) that contains the suffixes of the string $ 1$528...3Sk#, 
where $ and # are two different digits that do not appear in any of the strings $1, 
$2, +++, Sk. In each node of the suffix tree, we keep a list of all strings Si that 
are the start point of a suffix represented by an element node in that subtrie. 


Find the longest substring of S that appears at least m>1 times. 
This query can be answered in O(|S 1) time in the following way: 


(a) Traverse the suffix tree labeling the branch nodes with the sum of the label 
lengths from the root and also with the number of information nodes in the 
subtrie. 

(b) Traverse the suffix tree visiting branch nodes with element node count 2m. 
Determine the visited branch node with longest label length. 


Note that step (a) needs to be done only once. Following this, we can do 
step (b) for as many values of m as is desired. Also, note that when m = 2 we 
can avoid determining the number of element nodes in subtries. In a compressed 
trie, every subtrie rooted at a branch node has at least two element nodes in it. 


Find the longest common substring of the strings S and 7. 
This can be done in time O(1S 1+17 1) as below: 


(a) Construct a multiple string suffix tree for $ and 7 (i.c., the suffix tree for 
SST#). 

(b) Traverse the suffix tree to identify the branch node for which the sum of the 
label lengths on the path from the root is maximum and whose subtrie has 
at least one information node that represents a suffix that begins in S and at 
least one information node that represents a suffix that begins in 7. 


678 Digital Search Structures 


EXERCISES 


Draw the suffix tree for S = ababab#. 

2. Draw the suffix tree for S = aaaaaa#. 
Draw the multiple string suffix tree for $1 = abba, $2 = bbbb, and $3 = 
aaaa. 


12.5 TRIES AND INTERNET PACKET FORWARDING 


12.5.1 IP Routing 


In the Internet, data packets are transported from source to destination by a series 
of routers. For example, a packet that originates in New York and is destined for 
Los Angeles will first be processed by a router in New York. This router may 
forward the packet to a router in Chicago, which, in turn, may foward the packet 
to a router in Denver. Finally, the router in Denver may forward the packet to 
Los Angeles. Each router moves a packet one step closer to its destination. A 
router does this by examining the destination address in the header of the packet 
to be routed. Using this destination address and a collection of forwarding rules 
stored in the router, the router decides where to send the packet next. 

An Internet router table is a collection of rules of the form (P, NH), where 
where P is a prefix and NH is the next hop; NH is the next hop for packets whose 
destination address has the prefix P. For example, the rule (01 *,a) states that the 
next hop for packets whose destination address (in binary) begins with 0] is a. 
In IPv4 (Internet Protocol version 4), destination addresses are 32 bits long. So, P 
may be up to 32 bits long. In IPv6, destination addresses are 128 bits long and so, 
P may be up to 128 bits in length. 

It is not uncommon for a destination address to be matched by more than | 
tule in a commercial router table. In this case, the next hop is determined by the 
matching rule that has the longest prefix. So, for example, suppose that (01 *,a) 
and (0100*,6) are the only two rules in our router table that match a packet 
whose destination address begins with the bit sequence 0100. The next hop for 
this packet ts b. In other words, packet forwarding in the Internet is done by 
determining the longest matching-prefix. 

Although Internet router tables are dynamic in practice (i.e., the rule set 
changes in time; rules are added and deleted as routers come online and go 
offline), data structures for Internet router tables often are optimized for the 
search operation—given a destination address, determine the next hop for the 
longest matching-prefix. 
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12.5.2 1-Bit Tries 


A 1-bit trie is very similar to a binary trie. It is a tree-like structure in which 
each node has a left child, left data, right child, and right data field. Nodes at 
level / of the trie store prefixes whose length is f. If the rightmost bit in a prefix 
whose length is / is 0, the prefix is stored in the left data field of a node that is at 
level /; otherwise, the prefix is stored in the right dala field of a node that is at 
level /. At level i of a trie, branching is done by examining bit i (bits are num- 
bered from left to right beginning with the number 1) of a prefix or destination 
address. When bit i is 0, we move into the left subtree; when the bit is 1, we 
move into the right subtree. Figure 12.31{a) gives a set of 8 prefixes, and Figure 
12.31(b) shows the corresponding !-bit wie. 


Pl=10* 
P2=111* 
P3= 11001* 
P4=1* 
PS =0* 
P6 = 1000* 
P7 = 100000* N7 
P8 = 1000000* PB] 
(a) 8 prefixes (b) Corresponding i-bit trie 


Figure 12.31: Prefixes and corresponding 1-bit trie 


The height of a 1-bit trie is O(W), where W is the length of the longest prefix in 
the router table. Note that W < 32 for IPv4 tables and W < 128 for IPV6 tables. 
Note also that there is no place, in a I-bit trie, to store the prefix * whose length 
is zero. This doesn’t lead to any difficulty as this prefix matches every 
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destination address. In case a search of a 1-bit trie fails to find a matching prefix, 
the next-hop associated with * is used. : 

For any destination address d, all prefixes that match d lie on the search 
path determined by the bits of d. By following this search path, we may deter- 
mine the longest matching-prefix in O(W) time. Further, prefixes may be 
inserted/deleted in O{W) time. The memory required by a 1-bit trie is O(nW), 
where 7 is the number of rules in the router table. 

Although the algorithms to search, insert and delete using a 1-bit trie are 
simple and of seemingly tow complexity, O(W), the demands of the Internet 
make the 1-bit trie impractical. Using trie-like structures, most of the time spent 
searching for the next hop goes to memory acceses. Hence, when analyzing the 
complexity of trie data structures for router tables, we focus on the number of 
memory accesses. When a 1-bit trie is used, it may take us up to W memory 
accesses to determine the next hop for a packet. Recall that W < 32 for IPv4 and 
W $128 for IPv6. To keep the Intemet operating smoothly, it is necessary that 
the next hop for each packet be determined using far fewer memory accesses 
than W. In practice, we must determine the next hop using at most (say) 6 
memory accesses. 


12.5.3 Fixed-Stride Tries 


Since the trie of Figure 12.31(b) has a height of 7, a search into this trie may 
make up to 7 memory accesses, one access for each node on the path from the 
root to a node at level 7 of the trie. The total memory required for the 1-bit trie of 
Figure 12.31(b) is 20 units (each node requires 2 units, one for each pair of 
(child, data) fields). We may reduce the height of the router-table trie at the 
expense of increased memory requirement by increasing the branching factor at 
each node, that is, we use a multiway trie. The stride of a node is defined to be 
the number of bits used at that node to determine which branch to take. A node 
whose stride is s has 2° child fields (corresponding to the 2° possible values for 
the s bits that are used) and 2’ data fields. Such a node requires 25 memory units. 
In a fixed-stride trie (FST), all nodes at the same level have the same stride; 
nodes at different levels may have different strides. 

Suppose we wish to represent the prefixes of Figure 12.31(a) using an FST 
that has three levels. Assume that the strides are 2, 3, and 2. The root of the trie 
stores prefixes whose length is 2; the level two nodes store prefixes whose length 
is 5 (2 + 3); and level three nodes store prefixes whose length is 7 (2 + 3 + 2). 
This poses a problem for the prefixes of our example, because the length of some 
of these prefixes is different from the storeable lengths. For instance, the length 
of PS is I. To get around this problem, a prefix with a nonpermissible length is 
expanded to the next permissible length. For example, PS = 0* is expanded to 
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PSa = 00* and PSb = 01*. If one of the newly created prefixes is a duplicate, 
natural dominance rules are used to eliminate all but one occurrence of the 
prefix. For instance, P4 = [* is expanded to P4a = 10* and P4b = 11*. However, 
Pi = 10% is to be chosen over P4a = 10*, because Pt is a longer match than P4, 
So, P4a is eliminated. Because of the elimination of duplicate prefixes from the 
expanded prefix set, all prefixes are distinct. Figure 12.32(a) shows the prefixes 
that result when we expand the prefixes of Figure 12.31 to lengths 2, 5, and 7. 
Figure 12.32(b) shows the corresponding FST whose height is 3 and whose 
strides are 2, 3, and 2. 


P1 = 10* 000 { P6 | 000 
P2a = 11100* 001 00) 
P2b = 11101* oof - | 010 
P2c = 11110* Olt Ol 
P2d = 11111* 100 100 
P3 = 11001* 101 101 
P4=11* 110 110 
PSa = 00* pay 11 
P5b = 01* 

P6a = 10000* P8 00 
P6b = 10001* P? 01 
P7 = 100000* = 10 
P8 = 1000000* Pa 0 

(a) Expanded prefixes (b) Corresponding fixed-stride trie 


Figure 12.32: Prefix expansion and fixed-stride trie 


Since the trie of Figure 12.32(b) can be searched with at most 3 memory 
accesses, it represents a time performance improvement over the 1-bit trie of 
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Figure 12.31(b), which requires up to 7 memory references to perform a search. 
However, the space requirements of the FST of Figure 12.32(b) are more than 
that of the corresponding 1-bit tie. For the root of the FST, we need 8 fields or 4 
units; the two level 2 nodes require 8 units each; and the level 3 node requires 4 
units. The total is 24 memory units. 

We may represent the prefixes of Figure 12.31(a) using a one-level trie 
whose root has a stride of 7. Using such a trie, searches could be performed mak~- 
ing a single memory access. However, the one-level trie would require 27 = 128 
memory units. 

In the fixed-stride trie optimization (FSTO) problem, we are given a set P 
of prefixes and an integer k. We are to select the strides for a k-level FST in such 
a manner that the k-level FST for the given prefixes uses the smallest amount of 
memory. 

For some P, a k-level FST may actually require more space than a (k—1)- 
level FST. For example, when P = {00*, 01*, 10*, 11*}, the unique I-level FST 
for P requires 4 memory units while the unique 2-level FST (which is actually 
the 1-bit trie for P) requires 6 memory units. Since the search time for a (k—-1)- 
level FST is jess than that for a k-level tree, we would actually prefer (k-1)-level 
FSTs that take less (or even equal) memory over k-level FSTs. Therefore, in prac- 
tice, we are really interested in determining the best FST that uses at most k lev- 
els (rather than exactly k levels). The modified MSTO problem (MFSTO) is to 
determine the best FST that uses at most k levels for the given prefix set P. 

Let O be the 1-bit trie for the given set of prefixes, and let F be any k-level 
FST for this prefix set. Let 5), ++, 5, be the strides for P. We shall say that level 


Jj. 187 Sk, of F covers levels a, ---, b of O, where a=S5,+1 and b = Ss. So, 
t 


D 

level 1 of the FST of Figure 12.32(b) covers levels | and 2 of the 1-bit trie of Fig- 

ure $2.31(b). Level 2 of this FST covers levels 3, 4, and 5 of the 1-bit trie of Fig- 

ure 12.31(b); and level 3 of this FST covers levels 6 and 7 of the 1-bit trie. We 
f 


shall refer to levels ¢,= is. 1 Su <k as the expansion levels of O. The expan- 


t 
ston levels defined by the FST of Figure !2.32(b) are |, 3, and 6. 
Let nodes (i) be the number of nodes at level i of the 1-bit trie O. For the 
1-bit tne of Figure 12.31(a), nodes (1:7) = [1,1,2,2,2,1,1]. The memory required 


by F is Ynades (e,)*2'*. For example, the memory required by the FST of Fig- 


1 
ure 12.32(b) is nodes (1)*2* + nodes (3)*23 + nodes (6)*2? = 24, 

Let Tj.r) be the best (i.¢., uses least memory) FST that uses at most r 
expansion levels and covers levels | through j of the 1-bit trie O. Let CUj,r) be 
the cost (1... memory requirement) of T(j,r). So, T(W.k) is the best FST for O 
that uses at most k expansion levels and C(W,k) is the cost of this FST. We 
observe that the last expansion level in T(j.r) covers levels m + 1 through j of O 


Internet Packet Forwarding 683 


for some m in the range 0 through j ~ I and the remaining levels of this best FST 
define T(n,r—1). So, 


CGn= in {C (r-}enodes (me +i e2e*! LJ2tr>1 (2 


C(0,r) = Oand CG, 1) =2,j21 (12.2) 


Let M(j,r), r > 1, be the smallest m that minimizes 
C(m,r-1) + nodes (m+\)s2i-"*", 


in Bq. 12.1. Eqs. 12.1 and 12.2 result in an algorithm to compute C(W,k) in 
O(kW?). The M (j,r)s may be computed in the same amount of time while we are 
computing the C (j,r)s. Using the computed M values, the strides for the optimal 
FST that uses at most & expansion levels may be determined in an additional 
O(k) time. 


12.5.4 Variable-Stride Tries 


In a variable-stride trie (VST) nodes at the same level may have different strides. 
Figure 12.33 shows a two-level VST for the 1-bit trie of Figure 12.3]. The stride 
for the root is 2; that for the Jeft child of the root is 5; and that for the root’s right 
child is 3. The memory required by this VST is 4 (root) + 32 (left child of root) + 
8 (right child of root) = 44, 

Since FSTs are a special case of VSTs, the memory required by the best 
VST for a given prefix set P and number of expansion levels k is less than or 
equal to that required by the best FST for P and k. 

Let r-VST be a VST that has at most r jevels. Let Opr(N,r) be the cost 
{i.e., memory requirement) of the best r-VST for a I-bit trie whose root is N. The 
root of this best VST covers levels 1 through s of O for some s in the range 1 
through height (N) and the subtries of this root must be best (r—1)-VSTs for the 
descendents of N that are at level s + 1 of the subtree rooted at N. So, 


Opt(Nr)= min {2+ YF Opr(M,r-1)}, r>1 28) 
Isssheht(M)—— yed,.\tNy 


where D,(N) is the set of all descendents of N that are at level s of N. For exam- 
ple, D»(N) is the set of children of N and D3(N) is the set of grandchildren of N. 
height (N) is the maximum level at which the trie rooted at N has a node. For 
example, in Figure 12.31(b), the height of the trie rooted at NI is 7. When r=1, 
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00000 
00001 
00010 
00011 
00100 
00101 
00110 
ool 


11100 
11101 
11110 
11 


Figure 12.33: Two-level VST for prefixes of Figure 12.31(a) 
Opt (N, =2heisht (™), (12.4) 
Op(Nsr)= YL Opt(Mr)s>ir>1, 
AEDAN) 


and fet Opt(N, 1.r)=Oprt(N,r). From Eqs. 12.3 and 12.4, it follows that: 


Opt(N, Vr) = i 2 N,s+1,r-1)}, ve 
Tat ry exsist +OpttN,s+),r-1)},r > 1 (12.5) 
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and 


Opt(N, 1,1) = ao), (12.6) 
Fors > | and r > 1, we get 


Opt(N,s,r)= S Opt(M,r) 
MeD{N) 


= Opt{LeftChild(N),s—1,r) 


+ Opt (RightChild(N),s—1,r). (12.7) 


For Eq. 12.7, we need the following initial condition: 
Opt (null,*,*) =0 (12.8) 


For an n-rule router table, the |-bit trie O has O(nW) nodes. So, the number 
of Opt(*,*,*) values is O(nW?k). Each Opt(*,s,*), 5 > 1, value may be com- 
puted in O(1) time using Eqs. 12.7 and 12.8 provided the Opt values are com- 
puted in postorder. The Opr(*, 1,*) values may then be computed in O(W) time 
each using Eqs. 12.5 and 12.6. Therefore, we may compute Opr(R,k) = 
Opt(R, 1,k), where R is the root of O, in O(aW?k) time. If we keep track of the s 
that minimizes the right side of Eq. 12.5 for each pair (N,r), we can determine 
the strides of all nodes in the optimal k-VST is an additional O{nW) time. 


EXERCISES 
1, (a) Write a C++ function to compute C(j,r) for O<j < Wand 1srsk 
using Eqs. 12.1 and 12.2. Your function should compute M (j,r) as 
well. The complexity of your function should be O(kW?). Show that 
this is the case. 

(b) Write a C++ function that determines the strides of all levels in the 
best FST that has at most & levels. Your function should use the 
values computed in part (a). The complexity of your function should 
be Ok). Show that this is the case. 
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2, (a) Write a C++ function to compute Opt(N.s.r) fori ss<W.1Srs k 
and all nodes N of the I-bit trie O. You should use Eqs. 12.5 through 
12.8. Your function should compute S(N,r), which is the s value that 
minimizes the right side of Eq. 12.5, as well. The complexity of your 
function should be O(nW2k), where 1 is the number of rules in the 
router table. Show that this is the case. 


(b) Write a C++ function that determines the strides of all nodes in the 
best k-VST for O. Your function should use the S values computed in 
part (a). The complexity of your function should be O(nW). Show 
that this is the case. 
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http://www.om!.gov/TechResources/Human_Genome/home.html (Department of 
Energy's Web site for the human genomics project); and 
hup://merlin.mbcr.bem.tme.edu:800 1/ocd/Curric/welcome.html; (Biocomputing 
Hypertext Coursebook). 

Linear time algorithms to search for a single pattern in a given string can 
be found in most algorithm's texts. See, for example, the texts: Computer Algo- 
rithms, by E. Horowitz, $. Sahni, and S. Rajasekeran, Computer Science Press, 
New York, 1998 and Introduction to Algorithms, by T. Cormen, C. Leiserson, 
and R. Rivest, McGraw-Hill Book Company, New York, 1992. 

For more on suffix tree construction, see the papers: ‘‘A space economical 
suffix tree construction algorithm,’’ by E. McCreight, Journal of the ACM, 23, 2, 
1976, 262-272; ‘Fast string searching with suffix trees," by M. Nelson, Dr. 
Dobb's Journal, August 1996. and ‘*Suffix trees and suffix arrays,’’ by S. Aluru, 
in Handbook of data structures and applications, D. Mehta and S. Sahni, editors, 
Chapman & Hall/CRC, 2005. 

You can download C++ code to construct a suffix tree from 
hutp://www.ddj.com/ftp/1996/1996.08/suffix.zip. 

The use of fixed- and variable-stride tries for IP router tables was first pro- 
posed in the paper ‘Faster IP lookups using controlled prefix expansion,”’ by V. 
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